Bug 346230 - [clean up] certain combination of Clean Up settings messes up the code (ie. garbles it)
Summary: [clean up] certain combination of Clean Up settings messes up the code (ie. g...
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Text (show other bugs)
Version: 3.6   Edit
Hardware: All All
: P3 major with 1 vote (vote)
Target Milestone: 4.3 M6   Edit
Assignee: Noopur Gupta CLA
QA Contact:
URL:
Whiteboard:
Keywords: investigate
: 388300 (view as bug list)
Depends on:
Blocks: 401418
  Show dependency tree
 
Reported: 2011-05-18 08:30 EDT by John CLA
Modified: 2013-03-13 05:51 EDT (History)
6 users (show)

See Also:
daniel_megert: review+


Attachments
Organize Imports settings for project (87 bytes, text/plain)
2011-05-18 08:32 EDT, John CLA
no flags Details
I used `my formatter` profile from this list of all exported Formatters (58.92 KB, application/xml)
2011-05-18 08:33 EDT, John CLA
no flags Details
code templates used for current project (2.33 KB, application/xml)
2011-05-18 08:34 EDT, John CLA
no flags Details
the cleanup profile used for project and the one causing the garbling (3.74 KB, text/xml)
2011-05-18 08:35 EDT, John CLA
no flags Details
original sourcecode on which cleanup is to be applied (7.33 KB, text/plain)
2011-05-18 08:37 EDT, John CLA
no flags Details
same sourcecode after Clean Up (7.77 KB, text/plain)
2011-05-18 08:38 EDT, John CLA
no flags Details
ok there I did isolate it for you in this project called `new3` (25.76 KB, application/octet-stream)
2011-05-18 08:54 EDT, John CLA
no flags Details
Patch (1.54 KB, patch)
2013-02-20 00:48 EST, Noopur Gupta CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description John CLA 2011-05-18 08:30:02 EDT
Build Identifier: I20110514-0800

Causes me to lose data, in a way :) feel free to downgrade the prio as you disagree :P

for example this original code:
"
duration = initialDuration / speed;
"

this after Clean Up code:
"
this.duration = initialDura@Override
	tion / this.speed;
"

It actually messes it up even better when more cleanup options are enabled like so:
orig: http://pastebin.com/JCUqMW14
cleanup: http://pastebin.com/Fpe7ZZB1


Reproducible: Sometimes

Steps to Reproduce:
I cannot really reproduce this outside of my context, but well as it is here I've tried to remove some options which obviously had no effect on this bug happening, so:
I have this file here: http://pastebin.com/JCUqMW14
I have Project->Properties->Java Code Style->Clean Up->enabled and using this profile
Add 'this' qualifier to unqualified field accesses
Change non static accesses to static members using declaring type
Add final modifier to local variables
Add missing '@Override' annotations
Add missing '@Override' annotations to implementations of interface methods
Add missing '@Deprecated' annotations
(I'll probably include the exported xml of it, in attachments here)

All of the above must be set, else if one of them is not unset, the code garbling does not occur.

Also, Code Templates is disabled, Formatter is disabled, Organize Imports is disabled
and on Java Editor->Save Actions is disabled, hmm this means it uses Workspace one, I'll disable it and retry, ok disabled Window-Preferences->Java->Editor->Save Actions too, and also hmm the formatter used seems to be from Workspace then, and Code Templates and Organize Imports...
Ok, I then enable Java Editor->Save Actions for current project but without having anything selected on Perform the selected actions on save
then I enable Java Code Style->Code Templates for current project and I also enable there, Formatter for current project and also enable Organize Imports for current project ; I don't know how to factor these out of the problem, so they probably are affecting the bug

now just hover over the java file and select Source->Clean Up, then Use configured profiles (the profile above), press Next, a dialog shows:
"An error has occurred. See error log for more details.
No target edit provided."
and no preview is available
press Finish
and voila, the garbled code is here: http://pastebin.com/qTV0RZgi
Edit->Undo Cleanup actually works to revert this back, I previously used Team->Revert hmm

If any of the 4 big groups in cleanup profile is disabled then code garbling does not occur, thus it's the combination of all 4 (+ maybe formatter? and others) that do this
Comment 1 John CLA 2011-05-18 08:32:03 EDT
Created attachment 195957 [details]
Organize Imports settings for project
Comment 2 John CLA 2011-05-18 08:33:14 EDT
Created attachment 195958 [details]
I used `my formatter` profile from this list of all exported Formatters
Comment 3 John CLA 2011-05-18 08:34:00 EDT
Created attachment 195959 [details]
code templates used for current project
Comment 4 John CLA 2011-05-18 08:35:31 EDT
Created attachment 195960 [details]
the cleanup profile used for project and the one causing the garbling
Comment 5 John CLA 2011-05-18 08:37:22 EDT
Created attachment 195961 [details]
original sourcecode on which cleanup is to be applied
Comment 6 John CLA 2011-05-18 08:38:25 EDT
Created attachment 195962 [details]
same sourcecode after Clean Up
Comment 7 Olivier Thomann CLA 2011-05-18 08:52:50 EDT
I'll take a look.
Comment 8 John CLA 2011-05-18 08:54:30 EDT
Created attachment 195966 [details]
ok there I did isolate it for you in this project called `new3`

it's a rar file containing the project as it were in my workspace
all you need to do, supposedly, is just import it into your workspace and then move yourself in package explorer on the file AbstractCinematicEvent.java
in com.jme3.cinematic.events.AbstractCinematicEvent

and do Source->Clean Up
then go to line 76 for example and see this:
"
        this.duration = initialDura@Override
	tion / this.speed;
"
Comment 9 Olivier Thomann CLA 2011-05-18 13:56:33 EDT
I am getting this error as well.
org.eclipse.text.edits.MalformedTreeException: No target edit provided.
	at org.eclipse.text.edits.MoveSourceEdit.performConsistencyCheck(MoveSourceEdit.java:208)
	at org.eclipse.text.edits.TextEdit.traverseConsistencyCheck(TextEdit.java:873)
	at org.eclipse.text.edits.MoveSourceEdit.traverseConsistencyCheck(MoveSourceEdit.java:183)
	at org.eclipse.text.edits.TextEdit.traverseConsistencyCheck(TextEdit.java:869)
	at org.eclipse.text.edits.TextEdit.traverseConsistencyCheck(TextEdit.java:869)
	at org.eclipse.text.edits.TextEditProcessor.checkIntegrityDo(TextEditProcessor.java:176)
	at org.eclipse.text.edits.TextEdit.dispatchCheckIntegrity(TextEdit.java:743)
	at org.eclipse.text.edits.TextEditProcessor.performEdits(TextEditProcessor.java:151)
	at org.eclipse.ltk.core.refactoring.MultiStateTextFileChange.performChangesInSynchronizationContext(MultiStateTextFileChange.java:941)
	at org.eclipse.ltk.core.refactoring.MultiStateTextFileChange.access$2(MultiStateTextFileChange.java:932)
	at org.eclipse.ltk.core.refactoring.MultiStateTextFileChange$4.run(MultiStateTextFileChange.java:906)
	at org.eclipse.ui.internal.editors.text.UISynchronizationContext.run(UISynchronizationContext.java:34)
	at org.eclipse.core.internal.filebuffers.TextFileBufferManager.execute(TextFileBufferManager.java:629)
	at org.eclipse.ltk.core.refactoring.MultiStateTextFileChange.performChanges(MultiStateTextFileChange.java:918)
	at org.eclipse.ltk.core.refactoring.MultiStateTextFileChange.getPreviewDocument(MultiStateTextFileChange.java:771)
	at org.eclipse.ltk.core.refactoring.MultiStateTextFileChange.getPreviewContent(MultiStateTextFileChange.java:745)
	at org.eclipse.ltk.internal.ui.refactoring.TextEditChangePreviewViewer.setInput(TextEditChangePreviewViewer.java:209)
	at org.eclipse.ltk.internal.ui.refactoring.AbstractChangeNode.feedInput(AbstractChangeNode.java:99)
	at org.eclipse.ltk.internal.ui.refactoring.PreviewWizardPage.showPreview(PreviewWizardPage.java:598)
	at org.eclipse.ltk.internal.ui.refactoring.PreviewWizardPage.access$6(PreviewWizardPage.java:583)
	at org.eclipse.ltk.internal.ui.refactoring.PreviewWizardPage$7.selectionChanged(PreviewWizardPage.java:574)
	at org.eclipse.jface.viewers.Viewer$2.run(Viewer.java:164)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:49)
	at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
	at org.eclipse.jface.viewers.Viewer.fireSelectionChanged(Viewer.java:162)
	at org.eclipse.jface.viewers.StructuredViewer.updateSelection(StructuredViewer.java:2188)
	at org.eclipse.jface.viewers.StructuredViewer.setSelection(StructuredViewer.java:1725)
	at org.eclipse.jface.viewers.TreeViewer.setSelection(TreeViewer.java:1139)
	at org.eclipse.jface.viewers.Viewer.setSelection(Viewer.java:394)
	at org.eclipse.ltk.internal.ui.refactoring.PreviewWizardPage.setVisible(PreviewWizardPage.java:505)
	at org.eclipse.jface.wizard.WizardDialog.updateForPage(WizardDialog.java:1260)
	at org.eclipse.jface.wizard.WizardDialog.access$4(WizardDialog.java:1239)
	at org.eclipse.jface.wizard.WizardDialog$8.run(WizardDialog.java:1228)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
	at org.eclipse.jface.wizard.WizardDialog.showPage(WizardDialog.java:1226)
	at org.eclipse.jface.wizard.WizardDialog.nextPressed(WizardDialog.java:915)
	at org.eclipse.jface.wizard.WizardDialog.buttonPressed(WizardDialog.java:428)
	at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:624)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:240)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4163)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3752)
	at org.eclipse.jface.window.Window.runEventLoop(Window.java:825)
	at org.eclipse.jface.window.Window.open(Window.java:801)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizardOpenOperation$1.run(RefactoringWizardOpenOperation.java:181)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizardOpenOperation.run(RefactoringWizardOpenOperation.java:193)
	at org.eclipse.ltk.ui.refactoring.RefactoringWizardOpenOperation.run(RefactoringWizardOpenOperation.java:116)
	at org.eclipse.jdt.internal.ui.refactoring.actions.RefactoringStarter.activate(RefactoringStarter.java:38)
	at org.eclipse.jdt.internal.corext.refactoring.RefactoringExecutionStarter.startCleanupRefactoring(RefactoringExecutionStarter.java:251)
	at org.eclipse.jdt.internal.ui.actions.AllCleanUpsAction.performRefactoring(AllCleanUpsAction.java:78)
	at org.eclipse.jdt.internal.ui.actions.CleanUpAction.run(CleanUpAction.java:176)
	at org.eclipse.jdt.internal.ui.actions.CleanUpAction.run(CleanUpAction.java:108)
	at org.eclipse.jdt.ui.actions.SelectionDispatchAction.dispatchRun(SelectionDispatchAction.java:275)
	at org.eclipse.jdt.ui.actions.SelectionDispatchAction.run(SelectionDispatchAction.java:251)
	at org.eclipse.jface.action.Action.runWithEvent(Action.java:498)
	at org.eclipse.jface.action.ActionContributionItem.handleWidgetSelection(ActionContributionItem.java:584)
	at org.eclipse.jface.action.ActionContributionItem.access$2(ActionContributionItem.java:501)
	at org.eclipse.jface.action.ActionContributionItem$5.handleEvent(ActionContributionItem.java:411)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1053)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4163)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3752)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$9.run(PartRenderingEngine.java:944)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:860)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:87)
	at org.eclipse.ui.internal.Workbench$3.run(Workbench.java:542)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:522)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:123)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1386)

When you do: Source>Clean up..., and you click on Next, do you also get an error ?
It looks like the source on which the changes are applied is out of synch. @Override is inserted all over at wrong locations.
Comment 10 Olivier Thomann CLA 2011-05-18 15:18:32 EDT
Move to JDT/Text to find out where this error is coming from. This might cause the positioning error for all @Override annotations.
Comment 11 John CLA 2011-05-18 15:46:06 EDT
if you mean this error:
"An error has occurred. See error log for more details.
No target edit provided."

then yeah I also get it, as I did mention it in the first post ;)

Thanks for your work!
Comment 12 John CLA 2011-05-19 06:20:42 EDT
while this bug still exists, I no longer need it fixed asap, I've aborted doing the project that required it to be fixed. I also set its prio to normal (critical was way too exaggerated for this bug)

Thanks for all the people that are working on it!
Respectfully,
me
Comment 13 Dani Megert CLA 2011-05-19 09:04:56 EDT
Simple test case:

1. start new workspace
2. paste the following code into the Package Explorer:
-----------------------------------------------------------------
public abstract class AbstractCinematicEvent implements CinematicEvent {

    protected PlayState playState = PlayState.Stopped;
    protected LoopMode loopMode = LoopMode.DontLoop;

    public boolean internalUpdate() {
        return loopMode == loopMode.DontLoop;
    }

    public void stop() {
    }

    public void read() {
    	Object ic= new Object();
        playState.toString();
    }
}

interface CinematicEvent {
	public void stop();
	public boolean internalUpdate();
}
enum PlayState {
	Stopped
}
enum LoopMode {
	DontLoop
}
-----------------------------------------------------------------
3. clean up the file using the profile from attachment 195960 [details]
==> exception
Comment 14 Dani Megert CLA 2012-08-30 07:23:32 EDT
*** Bug 388300 has been marked as a duplicate of this bug. ***
Comment 15 Noopur Gupta CLA 2013-02-07 06:43:47 EST
(In reply to comment #13)
> Simple test case:

In the above test code, we get the exception (org.eclipse.text.edits.MalformedTreeException) for the expression: 'loopMode.DontLoop' in internalUpdate().

As per the attached clean up profile, 2 operations i.e. 'AddThisQualifierOperation' and 'ToStaticAccessOperation' need to be performed at 'loopMode.DontLoop'.
If we remove either one of these 2 operations from the profile or if we change 'loopMode.DontLoop' to 'LoopMode.DontLoop' or 'this.loopMode.DontLoop' before applying the attached profile, we do not get any exception and the code is also proper.

The edit structure which is created for the test case is:
{MultiTextEdit} [259,150] [undefined]
  {InsertEdit} [259,0] <<this.
  {MoveTargetEdit} [259,0]
  {DeleteEdit} [259,8]
    {MoveSourceEdit} [259,8]
  {InsertEdit} [271,0] <<LoopMode
  {DeleteEdit} [271,8]
    {MoveSourceEdit} [271,8]
  {RangeMarker} [366,0]
  {InsertEdit} [366,0] <<final
  {RangeMarker} [366,0]
  {InsertEdit} [366,0] << 
  {InsertEdit} [400,0] <<this.
  {MoveTargetEdit} [400,0]
  {DeleteEdit} [400,9]
    {MoveSourceEdit} [400,9]

The MoveTargetEdit associated with '{MoveSourceEdit} [271,8]' comes as '{MoveTargetEdit} [0,0]' i.e. :   
{DeleteEdit} [271,8]
    {MoveSourceEdit} [271,8] -> {MoveTargetEdit} [0,0]
Also, please not that it is not available as a child of '{MultiTextEdit} [259,150]'.    
 
After the copy is performed on the edit, we get:    
{DeleteEdit} [271,8]
    {MoveSourceEdit} [271,8] -> null
    
Hence, on applying this copied edit, we get the exception: "org.eclipse.text.edits.MalformedTreeException: No target edit provided".
Comment 16 Noopur Gupta CLA 2013-02-20 00:48:09 EST
Created attachment 227312 [details]
Patch

To summarize:

In the test case given in comment #13, we have the following statement:
	return loopMode == loopMode.DontLoop;
where, 'DontLoop' is a static field.

The cleanup profile has set 2 cleanup operations which are applicable to this statement:
1. Always use 'this' qualifier for non-static field accesses ('loopMode' in above case).
	-> This will lead to:
	return this.loopMode == this.loopMode.DontLoop;
2. Use declaring class as qualifier for all static accesses through instances. 
	-> This will lead to:
	return loopMode == LoopMode.DontLoop;
	
Hence, for 'loopMode' in 'loopMode.DontLoop', we have 2 operations.

The 1st one i.e. AddThisQualifierOperation, creates a NodeRewriteEvent for 'loopMode' through ASTRewrite#replace(...) and also creates a CopySourceInfo for 'loopMode'.
The 2nd one i.e. ToStaticAccessOperation, updates/overwrites the NodeRewriteEvent for 'loopMode' created above in ASTRewrite#replace(...) but does not remove the CopySourceInfo for 'loopMode' created earlier.

Hence, while conversion to TextEdits, MoveSourceEdit is also created with initial MoveTargetEdit[0,0] as a child of DeleteEdit and is not updated later.
The expected DeleteEdit should not have any child in case only ToStaticAccessOperation needs to be performed on that node.

The proposed fix removes the AddThisQualifierOperation for a particular node from the list of operations, in case ToStaticAccessOperation is being added on the same node. Hence, we get the DeleteEdit without any child for ToStaticAccessOperation on that node.
The edit structure created for the test case after the fix is:
{MultiTextEdit} [259,150] [undefined]
  {InsertEdit} [259,0] <<this.
  {MoveTargetEdit} [259,0]
  {DeleteEdit} [259,8]
    {MoveSourceEdit} [259,8]
  {InsertEdit} [271,0] <<LoopMode
  {DeleteEdit} [271,8]
  {RangeMarker} [366,0]
  {InsertEdit} [366,0] <<final
  {RangeMarker} [366,0]
  {InsertEdit} [366,0] << 
  {InsertEdit} [400,0] <<this.
  {MoveTargetEdit} [400,0]
  {DeleteEdit} [400,9]
    {MoveSourceEdit} [400,9]

Also verified that the fix works for AbstractCinematicEvent.java in project 'new3' available in comment#8.

Dani, please have a look.
Comment 17 Dani Megert CLA 2013-02-21 07:04:50 EST
(In reply to comment #16)

Nice explanation. This also means, requiring a fresh AST for each clean up would not fix the problem.

There are just two minor comments to the patch:
- We normally use "see bug ..." not "see bug: ..." (fixed that).
- I assume you know that your for loop is lucky to exit after the modification, 
  otherwise it would result in a ConcurrentModificationException. That's fine in 
  this case, but I just wanted to mention, that in general, one must not modify 
  the collection while iterating over it.


Fixed with http://git.eclipse.org/c/jdt/eclipse.jdt.ui.git/commit/?id=470ff6a9d1aefcd997dc6e2968d0e433923f0ebd
Comment 18 Noopur Gupta CLA 2013-02-21 07:17:16 EST
(In reply to comment #17)
Thanks Dani.

> This also means, requiring a fresh AST for each clean up
> would not fix the problem.
Correct.

> - We normally use "see bug ..." not "see bug: ..." (fixed that).
I will take care of this from next time.

> - I assume you know that your for loop is lucky to exit after the
> modification, otherwise it would result in a ConcurrentModificationException. 
Or I could have used it.remove() (the iterator's remove() function) in the other case.
Comment 19 Dani Megert CLA 2013-02-21 07:30:40 EST
Filed bug 401418 for a test case.
Comment 20 Dani Megert CLA 2013-02-21 07:31:09 EST
.
Comment 21 Martin Mathew CLA 2013-03-13 05:25:58 EDT
verifying.
Comment 22 Martin Mathew CLA 2013-03-13 05:51:09 EDT
Verified in I20130312-1000.