Bug 308000 - [formatter] Formatter is missing options regarding Annotation Newlines
Summary: [formatter] Formatter is missing options regarding Annotation Newlines
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.5.2   Edit
Hardware: PC Windows 7
: P3 enhancement with 1 vote (vote)
Target Milestone: 3.7 M2   Edit
Assignee: Frederic Fusier CLA
QA Contact:
URL:
Whiteboard:
Keywords: api
: 317784 (view as bug list)
Depends on:
Blocks: 318010
  Show dependency tree
 
Reported: 2010-04-02 11:57 EDT by Jason CLA
Modified: 2011-05-02 06:42 EDT (History)
6 users (show)

See Also:
Olivier_Thomann: review+


Attachments
Proposed patch (30.81 KB, patch)
2010-06-25 12:37 EDT, Frederic Fusier CLA
no flags Details | Diff
New proposed patch (31.21 KB, patch)
2010-08-11 08:50 EDT, Frederic Fusier CLA
no flags Details | Diff
example project (4.14 KB, application/zip)
2010-08-16 15:07 EDT, Markus Keller CLA
no flags Details
New proposed patch + additional test (61.84 KB, patch)
2010-08-20 11:08 EDT, Frederic Fusier CLA
no flags Details | Diff
Final proposed patch (67.08 KB, patch)
2010-08-31 10:47 EDT, Frederic Fusier CLA
no flags Details | Diff
Final proposed patch + doc (69.49 KB, patch)
2010-09-03 11:41 EDT, Frederic Fusier CLA
no flags Details | Diff
Additional patch (4.82 KB, patch)
2010-09-03 12:07 EDT, Frederic Fusier CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jason CLA 2010-04-02 11:57:20 EDT
Build Identifier: 20100218-1602

There are three options for how to deal with newlines with respect to annotations -- these are not enough.  I want a newline after a method annotation, but not after a field annotation.

Reproducible: Always

Steps to Reproduce:
1. Open Preferences
2. Go to Java -> Code Style -> Formatter
3. Edit a Profile
4. Go to the New Lines tab
5. There is an option for 'members', but it does not distinguish between 'fields' and 'methods'
Comment 1 Remy Suen CLA 2010-04-02 12:43:03 EDT
Issues and enhancements dealing with the Java formatter belongs in the JDT bucket.
Comment 2 Olivier Thomann CLA 2010-04-04 12:15:14 EDT
This cannot be done for 3.6 as it would require new API constants.
Deferring for 3.7.
Comment 3 michael.t CLA 2010-06-24 08:36:59 EDT
*** Bug 317784 has been marked as a duplicate of this bug. ***
Comment 4 Frederic Fusier CLA 2010-06-24 08:58:05 EDT
Olivier, do you agree to target this enhancement for 3.7?
Comment 5 Olivier Thomann CLA 2010-06-24 10:02:17 EDT
Sure.
Comment 6 Frederic Fusier CLA 2010-06-25 12:37:04 EDT
Created attachment 172780 [details]
Proposed patch

This patch adds options for new lines after annotations on:
 - types
 - fields
 - methods
 - constructors
 - packages

Not sure the last two ones were required... Of course, they can be easily removed if we do not want to introduce them.
Comment 7 Markus Keller CLA 2010-06-29 12:25:59 EDT
- Javadoc of DefaultCodeFormatterConstants#
FORMATTER_INSERT_NEW_LINE_AFTER_ANNOTATION_ON_MEMBER is missing:
	 * {@link #FORMATTER_INSERT_NEW_LINE_AFTER_ANNOTATION_ON_CONSTRUCTOR}

But I would drop the 'on_constructor' preference altogether. The setting on package makes sense, since it would otherwise be unspecified, but constructors are just a subspecies of methods, and I don't think it makes sense to single them out, but throw abstract, static, and instance methods into one bucket.


- Problem in jdt.core compatibility layer: With project-specific settings ...

org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_local_variable=insert
org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_member=do not insert
org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_parameter=do not insert

..., I now get new lines after annotations in members. AFAICS, the problem is that JavaProject#getOptions(boolean) drops options that are not in the optionNames set.

The formatter preference dialog also comes up with wrong settings in some cases (with and without the patch for bug 318010). We probably also need to tweak the ProfileVersioner. I'll look at that once the Core bugs are squashed.
Comment 8 Frederic Fusier CLA 2010-08-11 08:50:34 EDT
Created attachment 176332 [details]
New proposed patch

This patch removes the constructor preference.

For the compatibility issue, I observed that:
1) we got the same issue while deprecating the FORMATTER_INSERT_NEW_LINE_AFTER_ANNOTATION preference on 3.4 version and nobody complains about it since then...
2) I have added two specific tests which uses deprecated options and they behave correctly (see test727 and test728 in FormatterRegressionTests in the provided patch). JDT/Core behaves correctly in term of backward compatibility as the used deprecated option value is well set onto the new formatter options. 

Hence, IMO, the change needs to be done in ProfileVersioner while initializing the profiles. Deprecated options need to be converted similarly as they are in DefaultCodeFormatterOptions.setDeprecatedOptions(Map). Then the options map keys will  correct when got from the project or from JavaCore. Note that these options are internal since preferences are used in JDT/Core and so they are not supposed to contain any deprecated keys...
Comment 9 Markus Keller CLA 2010-08-16 15:07:41 EDT
Created attachment 176715 [details]
example project

In this example project, I20100810-0800 behaves properly (only inserts new line in local variable). But with the patch, I get newlines everywhere except for parameters. 

I think the problem of your tests is that they don't read the options from the preferences file, so they don't see the bug. See comment 7 paragraph -2.
Comment 10 Frederic Fusier CLA 2010-08-20 11:08:55 EDT
Created attachment 177102 [details]
New proposed patch + additional test

(In reply to comment #9)
> Created an attachment (id=176715) [details]
> example project
> 
> In this example project, I20100810-0800 behaves properly (only inserts new 
> line in local variable). But with the patch, I get newlines everywhere
> except for parameters. 
> 
> I think the problem of your tests is that they don't read the options from the
> preferences file, so they don't see the bug. See comment 7 paragraph -2.

I've added test729() in FormatterRegressionTest reading a formatter profile which has the old FORMATTER_INSERT_NEW_LINE_AFTER_ANNOTATION_ON_MEMBER (i.e. "org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_member") set to "do not insert". The result is the correct expected behavior: the formatter only inserts a new line in local variable.

So, again, the problem is not in JDT/Core which already has the mechanism for backward compatibility.

Debugging a little further, it seems that the problem is in ProfileVersioner.updateAndComplete(Map, int). As version 11 is the last up-to-date version, all keys which are not in the newSettings (i.e. keys got from the DefaultCodeFormatterConstants.getEclipseDefaultSettings() map) are removed without any preservation of the oldSettings value. Hence, the returned map from the updateAndComplete method contains the new keys ("..._on_field", "..._on_method", "..._on_package" and "..._on_type") with their default value ("insert") which was different than the old key value...

So, the conclusion is that the formatter profile needs to be upgraded to version 12 and a new conversion method version11to12(Map) added to preserve the org.eclipse.jdt.core.formatter.insert_new_line_after_annotation_on_member key value coming from older profiles.
Comment 11 Markus Keller CLA 2010-08-30 15:20:54 EDT
(In reply to comment #10)
Sure, the UI (including the ProfileVersioner) needs to be adapted to the new options. Until that's done, you can't use the UI to change or view options.

But it should be possible for a project to check in a .settings folder in 3.6, and then run with these settings in a 3.7 workspace with no changes. As long as the formatter properties page is not saved, the old settings should still work, but they don't. In this scenario, ProfileVersioner is never used. All we do is:
ToolFactory.createCodeFormatter(.., ToolFactory.M_FORMAT_EXISTING).format(..)

BTW: When trying this again, I noticed that the attached example project has a bug, which makes the project look like it's using the "Eclipse [built-in]" profile (but this doesn't affect my scenario where the user just formats): In .settings/org.eclipse.jdt.ui.prefs, line 3 should be:
formatter_profile=_bug308000
Comment 12 Frederic Fusier CLA 2010-08-31 10:47:13 EDT
Created attachment 177845 [details]
Final proposed patch

This patch addresses the backward compatibility issue described in comment 11 scenario. Note that to fix it, I needed to add an option migration method on JavaModelManager as it has also to be done while getting workspace options...

Olivier, could you have a look on this proposal and let me know your thoughts about it? TIA
Comment 13 Olivier Thomann CLA 2010-09-03 09:20:33 EDT
(In reply to comment #12)
> Olivier, could you have a look on this proposal and let me know your thoughts
> about it? TIA
+1. This should cover all possible cases for annotation new lines.
Comment 14 Frederic Fusier CLA 2010-09-03 11:41:42 EDT
Created attachment 178168 [details]
Final proposed patch + doc

Same patch + added documentation to build notes...
Comment 15 Frederic Fusier CLA 2010-09-03 11:42:09 EDT
(In reply to comment #14)
> Created an attachment (id=178168) [details]
> Final proposed patch + doc
> 
> Same patch + added documentation to build notes...

Released for 3.7M2 in HEAD stream.
Comment 16 Jason CLA 2010-09-03 11:49:00 EDT
Thanks a lot guys!

When will 3.7M2 be part of http://eclipse.org/downloads/ ?
Comment 17 Olivier Thomann CLA 2010-09-03 11:51:48 EDT
(In reply to comment #16)
> When will 3.7M2 be part of http://eclipse.org/downloads/ ?
3.7M2 is scheduled for September 17th
Comment 18 Frederic Fusier CLA 2010-09-03 12:07:02 EDT
Created attachment 178171 [details]
Additional patch

This additional patch moves the getCompatibleConstants method to JavaModelManager to avoid an unnecessary new API addition...
Comment 19 Frederic Fusier CLA 2010-09-03 12:08:05 EDT
(In reply to comment #18)
> Created an attachment (id=178171) [details]
> Additional patch
> 
Released for 3.7M2 in HEAD stream.
Comment 20 Satyam Kandula CLA 2010-09-14 12:34:18 EDT
(In reply to comment #19)
In the test cases and the example, I see package annotations on the package declaration in the file itself. As I understand the package annotations should be in package-info.java. Am I missing anything?
Comment 21 Markus Keller CLA 2010-09-14 12:48:27 EDT
I tried to acknowledge this in the preview with the comment:
// only allowed in package-info.java

I should probably expand this to:
// annotation on package is only allowed in package-info.java
=> See bug 325221.

But the formatter should not care about whether an annotation on a package is in package-info.java or not. Formatting is often done on fragments without clear context, and it doesn't hurt to format things correctly even if the compiler rejects them.
Comment 22 Olivier Thomann CLA 2010-09-14 15:41:15 EDT
Verified for 3.7M2 using I20100914-0100.
Remaining issue will be fixed as part of bug 324987.
Comment 23 Satyam Kandula CLA 2010-09-15 00:26:02 EDT
(In reply to comment #21)
> I tried to acknowledge this in the preview with the comment:
> // only allowed in package-info.java
> 
Thanks, I missed this in the preview.
Comment 24 Ayushman Jain CLA 2011-05-02 06:42:45 EDT
Verified for 3.7RC0 using build I20110428-0848.