Bug 179258 - simple reconcile starts problem finder - main thread waiting
Summary: simple reconcile starts problem finder - main thread waiting
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.3   Edit
Hardware: PC Windows XP
: P3 critical (vote)
Target Milestone: 3.3 M7   Edit
Assignee: Jerome Lanneluc CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2007-03-26 04:18 EDT by Dani Megert CLA
Modified: 2007-04-27 10:14 EDT (History)
1 user (show)

See Also:


Attachments
Proposed fix and regression test (12.81 KB, patch)
2007-03-27 05:22 EDT, Jerome Lanneluc CLA
no flags Details | Diff
New proposed fix and regression test (13.43 KB, patch)
2007-03-27 07:14 EDT, Jerome Lanneluc CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Dani Megert CLA 2007-03-26 04:18:01 EDT
3.3 M6

ICompilationUnit.reconcile(	ICompilationUnit.NO_AST, 
				false /* don't force problem detection */, 
				null /* use primary owner */, 
				null /* no progress monitor */);
)
should not create an AST but it does due to these lines of code:
  IProblemRequestor ownerProblemRequestor = 
                   this.workingCopyOwner.getProblemRequestor(workingCopy);
  this.resolveBindings |= problemRequestor != null && 
                   problemRequestor.isActive();
...
if (((this.reconcileFlags & ICompilationUnit.FORCE_PROBLEM_DETECTION) != 0)
                   || this.resolveBindings) {
                   ^^^^^^^^^^^^^^^^^^^^^^^^

Here's the stack trace:
Thread [main] (Suspended (breakpoint at line 172 in ReconcileWorkingCopyOperation))	
	ReconcileWorkingCopyOperation.makeConsistent(CompilationUnit, IProblemRequestor) line: 172	
	ReconcileWorkingCopyOperation.executeOperation() line: 85	
	ReconcileWorkingCopyOperation(JavaModelOperation).run(IProgressMonitor) line: 720	
	ReconcileWorkingCopyOperation(JavaModelOperation).runOperation(IProgressMonitor) line: 779	
	CompilationUnit.reconcile(int, int, WorkingCopyOwner, IProgressMonitor) line: 1169	
	CompilationUnit.reconcile(int, boolean, WorkingCopyOwner, IProgressMonitor) line: 1130	
	JavaModelUtil.reconcile(ICompilationUnit) line: 698	
	SelectionConverter.getElementAtOffset(IJavaElement, ITextSelection) line: 232	
	SelectionConverter.getElementAtOffset(JavaEditor, boolean) line: 151	
	SelectionConverter.getElementAtOffset(JavaEditor) line: 143	
	CompilationUnitEditor(JavaEditor).getAdapter(Class) line: 1767	
	CompilationUnitEditor.getAdapter(Class) line: 1682	
	Util.getAdapter(Object, Class) line: 106	
	ShowInMenu.getShowInSource(IWorkbenchPart) line: 212	
	ShowInMenu.getContext(IWorkbenchPart) line: 239	
	ShowInMenu.fillMenu(IMenuManager) line: 123	
	ShowInMenu.fill(Menu, int) line: 98	
	MenuManager.update(boolean, boolean) line: 665	
	MenuManager.update(boolean) line: 584	
	MenuManager.fill(Menu, int) line: 235	
	MenuManager.update(boolean, boolean) line: 665	
	MenuManager.handleAboutToShow() line: 395	
	MenuManager.access$1(MenuManager) line: 390	
	MenuManager$2.menuShown(MenuEvent) line: 416	
	TypedListener.handleEvent(Event) line: 234	
	EventTable.sendEvent(Event) line: 66	
	Menu(Widget).sendEvent(Event) line: 938	
	Menu(Widget).sendEvent(int, Event, boolean) line: 962	
	Menu(Widget).sendEvent(int) line: 943	
	Shell(Control).WM_INITMENUPOPUP(int, int) line: 3979	
	Shell(Control).windowProc(int, int, int, int) line: 3696	
	Shell(Decorations).windowProc(int, int, int, int) line: 1554	
	Shell.windowProc(int, int, int, int) line: 1752	
	Display.windowProc(int, int, int, int) line: 4342	
	OS.TrackPopupMenu(int, int, int, int, int, int, RECT) line: not available [native method]	
	Menu._setVisible(boolean) line: 227	
	Display.runPopups() line: 3699	
	Display.readAndDispatch() line: 3278	
	Workbench.runEventLoop(Window$IExceptionHandler, Display) line: 2331	
	Workbench.runUI() line: 2295	
	Workbench.access$4(Workbench) line: 2170	
	Workbench$4.run() line: 463	
	Realm.runWithDefault(Realm, Runnable) line: 289	
	Workbench.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 458	
	PlatformUI.createAndRunWorkbench(Display, WorkbenchAdvisor) line: 149	
	IDEApplication.start(IApplicationContext) line: 101	
	EclipseAppHandle.run(Object) line: 146	
	EclipseAppLauncher.runApplication(Object) line: 106	
	EclipseAppLauncher.start(Object) line: 76	
	EclipseStarter.run(Object) line: 356	
	EclipseStarter.run(String[], Runnable) line: 171	
	NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]	
	NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39	
	DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25	
	Method.invoke(Object, Object...) line: 585	
	Main.invokeFramework(String[], URL[]) line: 476	
	Main.basicRun(String[]) line: 416	
	Main.run(String[]) line: 1141	
	Main.main(String[]) line: 1116
Comment 1 Jerome Lanneluc CLA 2007-03-26 09:44:11 EDT
It looks like the working copy is not consistent before you call reconcile.
So it is expected that problems are being found since the problem requestor is active.
The ast that is created is an internal compiler ast (that is needed to find problems). No DOM AST is created in this case.

Or did I miss something ?
Comment 2 Dani Megert CLA 2007-03-26 11:30:59 EDT
Mmmh, when I debug I see that problems get reported even if problemRequestor.isActive() returns false as this is not tested here:
	// report problems
	if (this.problems != null && (((this.reconcileFlags & ICompilationUnit.FORCE_PROBLEM_DETECTION) != 0) || !wasConsistent)) {
		if (problemRequestor != null) {

In addition to the above code not checking for isActive() there is a race condition here: the text reconciler (thread) kicks in, enables the problem requestor and then the main thread simply wants to reconcile the structure without doing any more work but at that time the problem requestor is set to active. In the early days (up to < 3.2) this did not happen because we wrapped all calls to reconcile inside a synchronized(ICompilationUnit) {} to prevent parallel reconciles but then we got advised by JDT Core to remove this because it was claimed that JDT Core prevents two reconcile to run in parallel (this was debated at the JDT Summit 2005 in Zurich).
Comment 3 Jerome Lanneluc CLA 2007-03-27 04:36:12 EDT
(In reply to comment #2)
> Mmmh, when I debug I see that problems get reported even if
> problemRequestor.isActive() returns false 
Now I'm confused. In the original comment, the problem requestor was active (since this.resolveBindings was true).


> as this is not tested here:
>         // report problems
>         if (this.problems != null && (((this.reconcileFlags &
> ICompilationUnit.FORCE_PROBLEM_DETECTION) != 0) || !wasConsistent)) {
>                 if (problemRequestor != null) {
Right this is a bug and this needs to be fixed. Thanks for pointing at this.

> In addition to the above code not checking for isActive() there is a race
> condition here: the text reconciler (thread) kicks in, enables the problem
> requestor and then the main thread simply wants to reconcile the structure
> without doing any more work but at that time the problem requestor is set to
> active. In the early days (up to < 3.2) this did not happen because we wrapped
> all calls to reconcile inside a synchronized(ICompilationUnit) {} to prevent
> parallel reconciles but then we got advised by JDT Core to remove this because
> it was claimed that JDT Core prevents two reconcile to run in parallel (this
> was debated at the JDT Summit 2005 in Zurich).
Actually, I think we claimed that 2 reconciles could run in parallel as they are thread-safe. I still claim this is true. The problem is that the problem requestor is not thread-safe. Your implementation of isActive() should return true if running in the reconciler thread and it should return false if running in the main thread. This is easily achieved using a ThreadLocal field on the problem requestor to return the isActive state. Each thread can then set the isActive state indenpendantly of the other threads.

Comment 4 Dani Megert CLA 2007-03-27 04:52:49 EDT
>Now I'm confused. In the original comment, the problem requestor was active
>(since this.resolveBindings was true).
Yes. This is another issue. I tried to point that out in comment 2 with the words "In addition to the above" but obviously that wasn't clear enough - sorry about that.

OK, I can indeed easily fix on my side (did so for next I-build), but we weren't aware of this. What is the real use case for running two reconciles on the same CU with same working copy owner, especially when there can only be one problem requestor?
Comment 5 Dani Megert CLA 2007-03-27 05:01:57 EDT
Fixed the isActive() on our side for I20070327-0800.
Comment 6 Jerome Lanneluc CLA 2007-03-27 05:11:20 EDT
(In reply to comment #4)
> What is the real use case for running two reconciles on
> the same CU with same working copy owner, especially when there can only be one
> problem requestor?
In fact, we cannot prevent 2 reconciles (on the same working copy etc.) to run at the same time, since this would require to lock on the cu (as you used to do) and since there can be compilation participants, this calls for deadlocks (classic pattern where 2 locks are taken in different order).
Comment 7 Jerome Lanneluc CLA 2007-03-27 05:12:11 EDT
I will fix the problem with beginReporting() being called when the problem requestor is not active.
Comment 8 Dani Megert CLA 2007-03-27 05:16:02 EDT
k. We can mark this fixed then.
Comment 9 Jerome Lanneluc CLA 2007-03-27 05:22:33 EDT
Created attachment 62064 [details]
Proposed fix and regression test
Comment 10 Jerome Lanneluc CLA 2007-03-27 07:14:14 EDT
Created attachment 62072 [details]
New proposed fix and regression test

First fix was wrong since it would create an AST when:
- the working copy was consistent, 
- the problem requestor was active
- problem detection was not forced
- the ast level was JLS3
Comment 11 Jerome Lanneluc CLA 2007-03-27 10:29:16 EDT
Second fix and test released for 3.3M7 in HEAD.
Comment 12 Eric Jodet CLA 2007-04-27 10:06:09 EDT
Verified for 3.3 M7 using build I20070427-0010