Bug 236954 - [buildpath] Concurrency issues with classpath variable initialization
Summary: [buildpath] Concurrency issues with classpath variable initialization
Status: CLOSED WONTFIX
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2008-06-12 15:36 EDT by Yen Lu CLA
Modified: 2019-09-12 17:37 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Yen Lu CLA 2008-06-12 15:36:49 EDT
Build ID: 3.4RC1

Steps To Reproduce:
I have a classpathVariableInitializer extension defined in a plugin. The variable is added to a Java project after running a wizard. If I then shutdown and restart the workbench, the variable is now unbound and the project cannot be built and has error markers.

More information:
I think this is because my plugin is not loaded when I restart the workbench. Shouldn't the lazy loading of the classpath variable be done when the unbounded variable is detected?
Comment 1 Jerome Lanneluc CLA 2008-06-16 09:33:03 EDT
Indeed, your classpath variable initializer should be called again when you restart. Is there information in the .log file? Do you have more detailed steps so that we can reproduce?
Comment 2 Philipe Mulet CLA 2008-06-16 12:25:10 EDT
Maybe your plug-in is not seen as an extension, hence not found. Are you sure it is properly set up ?

This would result in the variable being unbound in the end.
Comment 3 Yen Lu CLA 2008-06-16 12:56:18 EDT
Following Jerome's hint about startup, I put breakpoints in my variable initializers and they are indeed called upon startup. As far as I can tell, they are calling the JavaCore.setClasspathVariable() APIs correctly too. In some cases, I'm seeing that the variable shows up in the preferences (Java => Build Path => Classpath Variables) but not all the time. When the variable does not show up there, the Java package explorer also does not show the jar as a referenced library node. I'm not sure I can create an actual test case but our code uses a single plugin to initialize multiple variables. The plugin is loading correctly as can be attributed to the fact that at least some of the variables appear to be initializing correctly. Note that I'm not getting unresolved variables now but this strange UI behavior with preferences and referenced libraries is now present.
Comment 4 Yen Lu CLA 2008-06-16 13:12:28 EDT
There is also a single initializer class being used for each of the variable initializer extensions.
Comment 5 Yen Lu CLA 2008-06-16 15:36:27 EDT
I think I know what might be happening. In our product, multiple threads end up invoking my classpath variable initializer. Here are some stack traces:

Thread 1:
CPVInitializer.initialize(String) line: 22	
JavaCore.getClasspathVariable(String) line: 2677	
JavaCore.getResolvedVariablePath(IPath) line: 3297	
JavaCore.getResolvedClasspathEntry(IClasspathEntry) line: 3212	
JavaProject.resolveClasspath(JavaModelManager$PerProjectInfo) line: 2540	
JavaProject.getResolvedClasspath() line: 1829	
DeltaProcessingState.initializeRoots(boolean) line: 229	
SetContainerOperation(JavaModelOperation).run(IProgressMonitor) line: 706	
Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 1800	
SetContainerOperation(JavaModelOperation).runOperation(IProgressMonitor) line: 775	
JavaCore.setClasspathContainer(IPath, IJavaProject[], IClasspathContainer[], IProgressMonitor) line: 4703	
JREContainerInitializer.initialize(IPath, IJavaProject) line: 69	
JavaModelManager.initializeContainer(IJavaProject, IPath) line: 2366	
JavaModelManager$13.run(IProgressMonitor) line: 2296	
Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 1800	
JavaModelManager.initializeAllContainers(IJavaProject, IPath) line: 2312	
JavaModelManager.getClasspathContainer(IPath, IJavaProject) line: 1674	
JavaCore.getClasspathContainer(IPath, IJavaProject) line: 2554	
JavaProject.resolveClasspath(JavaModelManager$PerProjectInfo) line: 2562	
JavaProject.getResolvedClasspath() line: 1829	
JavaModelManager.determineIfOnClasspath(IResource, IJavaProject) line: 928	
JavaModelManager.create(IFolder, IJavaProject) line: 813	
JavaModelManager.create(IResource, IJavaProject) line: 754	
JavaCore.create(IResource) line: 2439	
JavaSearchUtils.getIJavaElementsForClassPathEntries(IJavaProject) line: 385	
JavaSearchUtils.getIJavaElementsForClassPathEntriesForWorkspace() line: 362	

Thread2:
CPVInitializer.initialize(String) line: 22	
JavaCore.getClasspathVariable(String) line: 2677	
JavaCore.getResolvedVariablePath(IPath) line: 3297	
JavaCore.getResolvedClasspathEntry(IClasspathEntry) line: 3212	
JavaProject.resolveClasspath(JavaModelManager$PerProjectInfo) line: 2540	
JavaProject.getResolvedClasspath() line: 1829	
DeltaProcessingState.initializeRoots(boolean) line: 229	
SetContainerOperation(JavaModelOperation).run(IProgressMonitor) line: 706	
Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 1800	
SetContainerOperation(JavaModelOperation).runOperation(IProgressMonitor) line: 775	
JavaCore.setClasspathContainer(IPath, IJavaProject[], IClasspathContainer[], IProgressMonitor) line: 4703	
JREContainerInitializer.initialize(IPath, IJavaProject) line: 69	
JavaModelManager.initializeContainer(IJavaProject, IPath) line: 2366	
JavaModelManager$13.run(IProgressMonitor) line: 2296	
Workspace.run(IWorkspaceRunnable, ISchedulingRule, int, IProgressMonitor) line: 1800	
JavaModelManager.initializeAllContainers(IJavaProject, IPath) line: 2312	
JavaModelManager.getClasspathContainer(IPath, IJavaProject) line: 1674	
JavaCore.initializeAfterLoad(IProgressMonitor) line: 3398	
InitializeAfterLoadJob$RealJob.run(IProgressMonitor) line: 35	
Worker.run() line: 55	

My classpath variable initializer, CPVInitializer, is invoked through a JavaCore.getClasspathVariable() call. In CPVInitializer, I also call JavaCore.getClasspathVariable() as part of an attempt to check whether or not I really should be initializing. Should I just let the JavaCore.setClasspathVariable() calls go through or is there a better way in the initializer to check whether or not a variable is already initialized in this situation?



Comment 6 Jerome Lanneluc CLA 2008-06-17 05:22:57 EDT
ClasspathVariableInitializer.initialize(String) should be called exactly once per classpath variable. So you don't need to call getClasspathVariable() to know if it is initialized. In fact, this call will cause the initialization of the variable.

You should also call JavaCore.setClasspathVariable(...) for the variable being initialized *only*.
Comment 7 Philipe Mulet CLA 2008-06-17 05:48:45 EDT
Re: comment 5
It indeed feels strange that the same variable initializer would be called twice, as this is not possible in theory.

Normally, concurrent requests should cause the first request to trigger initialization, and the second request to fetch the previous session value until the end of the initialization.
Comment 8 Philipe Mulet CLA 2008-06-17 05:52:26 EDT
Jerome - look at JavaCore@line:2655
There could be a concurrency issue, if both threads asked #variableGet(...) simulteaneously, they would both think they should initialize. Hence two parallel invocations of the variable initializer (I think).
Comment 9 Yen Lu CLA 2008-06-17 09:30:29 EDT
I have changed the abstract to match what the issue really is. For now, I have put my own synchronization blocks to work around the problem.
Comment 10 Jerome Lanneluc CLA 2008-06-25 11:01:34 EDT
(In reply to comment #8)
> Jerome - look at JavaCore@line:2655
> There could be a concurrency issue, if both threads asked #variableGet(...)
> simulteaneously, they would both think they should initialize. Hence two
> parallel invocations of the variable initializer (I think).
> 
Actually, the 'initialization in progress' marker is a ThreadLocal variable. So per design, 2 threads are allowed to initialize the same variable. I believe this is to avoid deadlocks. So I was wrong in comment 6. I should have said that a variable is initialized at most once in a given thread.
Comment 11 Jerome Lanneluc CLA 2008-09-02 05:09:53 EDT
Re-reading this bug, I still don't understand what the issue is/was. Why was the variable not initialized on startup? Was it because the 2 parallel initializations caused the client CPVInitializer to missbehave? Yen I would appreciate some more explanation on the resolution of this bug.
Comment 12 Yen Lu CLA 2008-09-03 09:10:55 EDT
The issue was that there were two threads trying to go through the classpath variable initializer as you can see from my stack traces. Philippe's comments seem to indicate that this should not be happening. With the logic we had within our initializer, the initialization would fail. I have put synchronization blocks inside the initializer to ensure thread safety. Shouldn't variable initialization already be thread safe?
Comment 13 Philipe Mulet CLA 2008-09-03 11:22:19 EDT
Actually, I was mistaken. Blocking initialization can cause deadlocks, this is why we let them proceed in parallel; at most once per thread. Normally unless 2 threads were competing for initialization, an initializer would only perform once per session.

I suspect that adding synchronization in your code is quite dangerous, but assuming you are not calling out within synchronized code, it may be fine. Please ensure your initializer is quick then.
Comment 14 Yen Lu CLA 2008-09-03 11:35:05 EDT
Ok, there are definitely two JDT related threads vying for initialization. If I should not be using a synchronized block within my initializer, is there another way I can change my code to guarantee thread safety without introducing the possibility of deadlocks?
Comment 15 Jerome Lanneluc CLA 2008-09-04 04:41:57 EDT
Could you explain what thread safety you're trying to achieve? Isn't the initializer initializing with the same value if call twice in different threads?

Also re-reading comment 5, I don't think I was clear in my answer. When CPVInitializer.initialize(String) is called, it should not check if the variable is already initialized. It should just call JavaCore.setClasspathVariable(...) for the variable it is asked to initialize. Failing to do so, you may end-up with unbound variables.
Comment 16 Eclipse Genie CLA 2019-09-12 17:37:07 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.