Bug 324987 - [formatter] API compatibility problem with Annotation Newline options
Summary: [formatter] API compatibility problem with Annotation Newline options
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: 3.7 RC1   Edit
Assignee: Markus Keller CLA
QA Contact:
URL:
Whiteboard:
Keywords: api
Depends on: 345003
Blocks: 318010
  Show dependency tree
 
Reported: 2010-09-10 12:06 EDT by Markus Keller CLA
Modified: 2011-05-12 23:02 EDT (History)
6 users (show)

See Also:
Olivier_Thomann: review+


Attachments
Proposed patch (26.22 KB, patch)
2010-09-13 12:18 EDT, Frederic Fusier CLA
no flags Details | Diff
Proposed fix + regression tests (29.79 KB, patch)
2010-09-13 12:42 EDT, Olivier Thomann CLA
no flags Details | Diff
Fix 2 (26.49 KB, patch)
2011-05-09 09:17 EDT, Markus Keller CLA
no flags Details | Diff
Updated regression tests (6.19 KB, patch)
2011-05-09 13:11 EDT, Olivier Thomann CLA
no flags Details | Diff
Fix 4 (43.29 KB, patch)
2011-05-10 10:15 EDT, Markus Keller CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Markus Keller CLA 2010-09-10 12:06:35 EDT
Follow-up to bug 308000

We have a compatibility problem with 3.6 clients (e.g. our old UI). A 3.6 client expects IJavaProject#getOption(..) and #getOptions(..) to return values for the deprecated formatter options, but these are removed now.
Comment 1 Olivier Thomann CLA 2010-09-10 12:11:14 EDT
Should be fixed for 3.7M2.
Comment 2 Frederic Fusier CLA 2010-09-13 12:18:46 EDT
Created attachment 178758 [details]
Proposed patch

This patch addresses the options compatibility issue in a global manner. First it keeps deprecated options in the map. Second, it tries to migrate them automatically when storing the values into the preferences.

That has the advantage of avoiding to keep old entries in the preferences but ensure that old API clients will not lose their options values when calling the JDT/Core *options* methods...

I think I have added all possible project and workspaces configurations variations with a deprecated option to verify that we won't have any longer options backward compatibility issues.
Comment 3 Frederic Fusier CLA 2010-09-13 12:19:10 EDT
Olivier, please review, thanks
Comment 4 Olivier Thomann CLA 2010-09-13 12:41:38 EDT
Patch looks good.
I'll post a patch that adds two new tests and that adjusts the copyrights.
Comment 5 Olivier Thomann CLA 2010-09-13 12:42:18 EDT
Created attachment 178760 [details]
Proposed fix + regression tests

Add two new tests + change one copyright.
Everything else is the same.
Comment 6 Frederic Fusier CLA 2010-09-13 13:23:38 EDT
(In reply to comment #5)
> Created an attachment (id=178760) [details]
> Proposed fix + regression tests
> 
> Add two new tests + change one copyright.
> Everything else is the same.

Thanks Olivier :-)

Released for 3.7M2 in HEAD stream.
Comment 7 Markus Keller CLA 2010-09-13 14:15:32 EDT
Sorry, I just tried this patch, but now nothing works for me any more. When I revert bug 318010, then I'd expect that the old UI still works, but it doesn't. And with everything from HEAD, I can change the new options, but when I then try to format a CU, it only considers the deprecated options but not the new ones.

I think the safest option for M2 is to revert this patch.
Comment 8 Olivier Thomann CLA 2010-09-13 14:28:31 EDT
Reverted the fix. Frederic, please investigate using the latest patch.
Thanks.
Moving to M3 for safety.
Comment 9 Markus Keller CLA 2010-09-13 14:42:37 EDT
It would also help if the formatter constants said more clearly what happens when both old and new (or only old) options are set. A further complication is that options are inherited from the instance scope.
Comment 10 Frederic Fusier CLA 2010-09-14 04:52:11 EDT
Markus,

I'm still missing clear scenarios highlighting the problems you found to make us reverting the patch. From my point of view, the classic scenarios were:
1) an client using options API methods with deprecated options
2) read deprecated preferences from an old project/workspaces
As I said those two scenarios were covered by the proposed patch.

So could you be more precise on the following points:
1) what does exactly mean: "reverting 318010"? Does it match a scenario that a common user may encounter? If so, could you describe it?
2) how both old and new preferences could be set at the same time? As the proposed patch does not persist the deprecated preferences, I was not able to find a use case where that can happen... I agree that new and deprecated *options* can exist simultaneously in the map but they are supposed to have the same value. Again, if you have a precise scenario showing that the options can be desynchronized and screwed up the formatter, then please describe it to help me to reproduce and fix it.

TIA
Comment 11 Frederic Fusier CLA 2010-11-26 06:46:28 EST
I think this bug should be fixed for 3.7...

As I'm a little bit less busy with Jazz those days, would it be possible to restart work on it?

TIA
Comment 12 Markus Keller CLA 2010-11-29 06:39:55 EST
Sorry, we're in the week before M4 now, and I won't have time to get this done for M4. Scheduling for early M5.

The main 3 scenarios we should support are these:
a) pure 3.7
b) Eclipse 3.7, but project with 3.6 settings
c) Eclipse 3.6, but project with 3.7 settings (will not understand the new options, but should use reasonable values for the deprecated options)

(c) requires that we still store the old options in 3.7.

Parts of the problem could also be in the current JDT/UI code. We don't use the setOption* methods but set the preferences directly in the store.
Comment 13 Frederic Fusier CLA 2011-01-08 10:52:20 EST
(In reply to comment #12)
> Sorry, we're in the week before M4 now, and I won't have time to get this done
> for M4. Scheduling for early M5.
> 
> The main 3 scenarios we should support are these:
> a) pure 3.7
> b) Eclipse 3.7, but project with 3.6 settings
> c) Eclipse 3.6, but project with 3.7 settings (will not understand the new
> options, but should use reasonable values for the deprecated options)
> 
> (c) requires that we still store the old options in 3.7.
> 
> Parts of the problem could also be in the current JDT/UI code. We don't use the
> setOption* methods but set the preferences directly in the store.

Unless I miss something, a) and b) are already covered by the proposed patch.
And, unfortunately, I'm not sure to understand the real use case of c). Maybe this is related to comment 7 when you tried to "revert 318010"... So, as I was asking in comment 10, Markus, could you be a little bit more precise on this scenario and give me clear steps to help me to understand it and reproduce it easily?

As I'm now 100% on Jazz, I won't have too much time to work on this problem (only extra hours to be honest...;-)). Hence any hints which could help me to save time to figure out what is the exact compatibility problem here will be more than welcome :-)

TIA
Comment 14 Olivier Thomann CLA 2011-01-19 15:38:20 EST
Removing target milestone.
Comment 15 Frederic Fusier CLA 2011-02-06 06:26:24 EST
Ping...

Markus, I guess this needs to be fixed before M6. So, could you please answer my comment 10 and comment 13? As I said in comment 13, I won't have too much time to spend on this regarding my Jazz work overload, hence the sooner I have more information about the real issue for this bug, the better it is for me to plan to work on a possible fix...

TIA
Comment 16 Ayushman Jain CLA 2011-05-02 06:07:02 EDT
Too late for 3.7, but I think we should target 3.8. Olivier, what do you think?
Comment 17 Olivier Thomann CLA 2011-05-02 08:05:46 EDT
I would close as WONTFIX since once 3.7 is done, there is no need to work on this anymore.
Comment 18 Markus Keller CLA 2011-05-04 11:17:28 EDT
I found a compatibility problem when moving a project (org.eclipse.jdt.junit) to 3.7. Looks like we currently keep both the old and the new options in the .prefs file, and the old options wins.

After disabling all options in the UI, I still get newlines after annotations. I'm investigating.
Comment 19 Markus Keller CLA 2011-05-09 09:17:50 EDT
Created attachment 195065 [details]
Fix 2

There was a bug in the UI (bug 345003), which prevented profiles from being written into the .prefs file right after changing the profile version. I think I got tricked by that problem when I reopened this bug.

But after that bug is fixed and a formatter profile is successfully changed, we still have a compatibility issue in HEAD: When a project contains a deprecated formatter.insert_new_line_after_annotation_on_* option and one of the replacement options, then the deprecated option wins. This is bad if we want to keep at least some compatibility with old Eclipse versions in the project options (think of a shared project that is used with 3.6 and 3.7).

The 3.7 UI just leaves the old options in the file and adds the new options. In this situation, the new option should win and the old option should only be used as a fallback. This is broken both in HEAD and with Frederic's fix.

I think Frederic's fix makes sense and is useful for clients that use the jdt.core APIs to read/write preferences (including the code formatter itself). The code formatter UI currently reads/writes the preferences directly via the Eclipse preference store, so we are not directly affected by the original patch.

This patch (Fix 2) is Frederic's patch with the following changes:
- DefaultCodeFormatterOptions lets the new options win over the deprecated ones

- JavaModelManager and JavaProject's getOption methods use
	String compatibleOption = compatibleOptions[co];
	if (!options.containsKey(compatibleOption))
		options.put(compatibleOption, propertyValue);
... instead of ...
	options.put(compatibleOptions[co], propertyValue);

- doesn't include the tests
Comment 20 Markus Keller CLA 2011-05-09 09:20:00 EDT
Olivier (or Frederic, if you're still in the loop), could you please review Fix 2? I would release this for RC1.
Comment 21 Olivier Thomann CLA 2011-05-09 13:10:04 EDT
The patch looks good.
However this is still an issue for users that want to share the same project between 3.6 and 3.7.
Also the tests need to be updated as they are using mixed options (old and new) and expect the old one to win.
Comment 22 Olivier Thomann CLA 2011-05-09 13:11:13 EDT
Created attachment 195105 [details]
Updated regression tests
Comment 23 Markus Keller CLA 2011-05-10 10:15:55 EDT
Created attachment 195218 [details]
Fix 4

Final patch against HEAD including all tests.
Comment 24 Markus Keller CLA 2011-05-10 15:48:07 EDT
Fixed in HEAD and BETA_JAVA7.
Comment 25 Jay Arthanareeswaran CLA 2011-05-12 23:02:49 EDT
Verified for 3.7RC1 with build I20110511-0800.