Summary: | Deadlock between indexing and classloading | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | [Eclipse Project] Equinox | Reporter: | Stephan Herrmann <stephan.herrmann> | ||||||
Component: | Framework | Assignee: | equinox.framework-inbox <equinox.framework-inbox> | ||||||
Status: | RESOLVED WONTFIX | QA Contact: | |||||||
Severity: | major | ||||||||
Priority: | P3 | CC: | frederic_fusier, jerome_lanneluc, tjwatson | ||||||
Version: | 3.3.1 | Keywords: | needinfo | ||||||
Target Milestone: | --- | ||||||||
Hardware: | Other | ||||||||
OS: | Linux | ||||||||
Whiteboard: | |||||||||
Attachments: |
|
Description
Stephan Herrmann
2008-01-19 09:15:59 EST
I tried to reproduce this issue and never was able to. Two questions about the given stack trace: 1) it seems to have strange lines: - JavaCore.java line: 4708 - JavaCore.java line: 4538 - JavaCore.java line: 1590 and none of these lines matches java code in JavaCore.java R3_3_1 version... How did you get it and are your sure you were using the referred build ID? 2) I do not see another thread requesting a class loader which may explain the lock. Can you attach to this bug the complete stack trace? Created attachment 87434 [details] corrected stacktrace I'm sorry about incomplete/incorrect information! I forgot that we do have a few source level changes in classes JavaCore, JavaProject and JavaModelManager. I strongly believe that these changes are unrelated to the observed deadlock, but line numbers are broken wrt. any version in your CVS. Sorry (plus the debugger swallowed some method names/signatures due to our use of JSR 045, but that's not a bug..) I currently sit at a different computer, so I cannot easily reproduce, but I tried to translate broken line number information back to a clean state as of v_780. Please excuse if I made mistakes during this manual correction. > 2) I do not see another thread requesting a class loader which may explain > the lock. Can you attach to this bug the complete stack trace? Sorry again for the incomplete stacktrace. Let me try to explain better (for the lack of a more complete stacktrace, currently): Due to the global locking strategy, thread [Java indexing] tries to acquire the global classloading lock using BundleLoader.lock(Object). However, this global lock is owned by thread [main] because startActivator at the bottom of stack [main] is called from within class-loading. Stackframes of thread [main] mentioning BundleLoader omitted at "... more ..." :( I did hope you could quickly confirm that calling JavaCore-methods from within an Activator might be dangerous. If you do want to reproduce I suggest the following (perhaps not telling much news here): + In the launch config pass option "-Dosgi.classloader.singleThreadLoads=true" as a vmarg. + have a workspace with some indexing to be done a startup + set a breakpoint at the first line of IndexManager.reset() => once hit, we have the lock on IndexManager + set another breakpoint at UMLPlugin.start() (or whatever client you use to trigger setOptions from) => once hit, this thread has the classloading lock + Now launch to run up-to these two breakpoints + now set a breakpoint at JobManager.discardJobs(String) and resume [main], which should eventually hit the new breakpoint, but should not be able to enter the "synchronized(this)" block, if "this" is the IndexManager + resume thread [Java indexing] and hope it needs to do some class loading. Do you see it? Created attachment 88076 [details]
stacktrace showing reproducable deadlock in jdt.debug.tests
I attach a stacktrace that I get reproducably when running jdt.debug.tests
in an unmodified Eclipse 3.3.0. Slightly different situation by symptoms
appear to be the same as before.
Steps for reproducing the stacktrace I just attached: 1. Clean Eclipse 3.3 with two projects in the workspace: org.eclipse.test.performance org.eclipse.jdt.debug.tests 2. Launch configuration (JUnit Plugin Tests) with these settings Project: org.eclipse.jdt.debug.tests Test class: org.eclipse.jdt.debug.tests.AutomatedSuite Run an application: org.eclipse.ui.ide.Workbench VM Arguments: -Dosgi.classloader.singleThreadLoads=true Everything else at defaults. 3. Set the following line breakpoints (maybe not all are needed, but this set is dead-safe): ContentDescriptionManager [line: 438] - setCacheState(byte) IndexManager [line: 255] - getIndex(IPath, IPath, boolean, boolean) LaunchingPlugin [line: 479] - start(BundleContext) SourceIndexer [line: 45] - indexDocument() Debug the above launch configuration, see the debugger stop at breakpoints, after waiting a little, resume, and watch the process getting locked in a state, where some threads are actively looping a dead loop, i.e., waiting for an unreachable state. Stacktraces are a little different from previous stacks, but I see these common ingredients: + Indexer is working, trying to load a class, blocked + Other plugin's Activator (here LaunchingPlugin) calls JavaCore.setOptions, which tries to schedule a Job, blocked while blocking class loading. The part I don't see is how Job scheduling contributes to the dead state. I can reproduce with 3.3 but not with 3.4M4. So, some changes in the Equinox framework seemed to have fix this issue => move to Equinox/Framework component to identify which one and set as duplicate... So far I have not been able to reproduce on either 3.3 or 3.4. I'm not aware of a fix that would have helped with this deadlock. OK, I found a significant difference in HEAD: three lines deleted from org.eclipse.jdt.debug.tests.AbstractDebugTest (at the end of <init>): if (!(this.getClass() == ProjectCreationDecorator.class) && !getJavaProject().exists()) { new TestSuite(ProjectCreationDecorator.class).run(new TestResult()); } If you re-add these lines, the deadlock can still be reproduced in 3.4M6. Run to the following breakpoints: thread main: LaunchingPlugin.start(BundleContext) line: 472 thread Java indexing: IndexManager.getIndex(IPath, IPath, boolean, boolean) line: 265 Try to step the main thread past the invocation of JavaCore.setOptions(optionsMap); This thread will hang while trying to schedule an indexer job. Now continue stepping the indexer thread, which will cause loading some more classes (I see org.eclipse.jdt.internal.core.search.indexing.ReadWriteMonitor) which blocks in BundleLoader.lock(Object) line: 1158 (Note that I'm running with singleThreadLoads=true). Does this help you reproduce in 3.4? Given that osgi.classloader.singleThreadLoads is mostly dead (cf. bug 227587), I assume this issue became irrelevant, too, since the deadlock was observed only with osgi.classloader.singleThreadLoads=true Should it then be closed as WONTFIX? closing as per previous comment. |