Community
Participate
Working Groups
I've seen this only once, but I suspect this is something related to the recent changes (I've just updated to the recent I- build). Start Eclipse fails after selection of the existing workspace in the dialog, with this error: eclipse.buildId=4.7.0.I20161101-0800 java.version=1.8.0_112 java.vendor=Oracle Corporation BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=de_DE Command-line arguments: -os win32 -ws win32 -arch x86_64 org.eclipse.core.jobs Error Sat Nov 05 08:58:03 CET 2016 An internal error occurred during: "Initializing workspace.". java.lang.IllegalStateException: No instance data can be specified. at org.eclipse.core.internal.runtime.DataArea.assertLocationInitialized(DataArea.java:50) at org.eclipse.core.internal.runtime.DataArea.getStateLocation(DataArea.java:124) at org.eclipse.core.internal.runtime.InternalPlatform.getStateLocation(InternalPlatform.java:539) at org.eclipse.core.runtime.Plugin.getStateLocation(Plugin.java:289) at org.eclipse.core.internal.resources.LocalMetaArea.<init>(LocalMetaArea.java:59) at org.eclipse.core.resources.ResourcesPlugin$1.run(ResourcesPlugin.java:513) at org.eclipse.core.runtime.jobs.Job$2.run(Job.java:186) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
This is a regression from bug 501997, commit 8d90fb030df310a974aae5d27d0bc2610c1a14ad.
I think the error could happen without patch for bug 501997. IIUC, this exception occurs because the Location service of type "osgi.configuration.area" is not available yet. I don't see how running in the initialization thread would help about the initialization order. Maybe there is a thread visibility issue... Did you managed to reproduce it?
(In reply to Mikaël Barbero from comment #2) > I think the error could happen without patch for bug 501997. IIUC, this > exception occurs because the Location service of type > "osgi.configuration.area" is not available yet. I don't see how running in > the initialization thread would help about the initialization order. Maybe > there is a thread visibility issue... > > Did you managed to reproduce it? No, but I've never seen this error before, and I start Eclipse pretty often :-)
(In reply to Mikaël Barbero from comment #2) > I think the error could happen without patch for bug 501997. IIUC, this > exception occurs because the Location service of type > "osgi.configuration.area" is not available yet. I don't see how running in > the initialization thread would help about the initialization order. Maybe > there is a thread visibility issue... Yes, this is strange, it can be of course that the patch uncovered existing threading issue with some shared states in equinox or elsewhere, or that the patch is entirely unrelated. However, the possibility this is just co-incidence, that such (seldom) critical startup problem occurs exact in the same place where we recently changed code is low. The only additional information I have is that this happened for me with my Oomph-based Eclipse, installed via Oomph. Not sure if this is related or not, I just recently switched to this one (and now back to default SDK after few issues). Oomph-based install is a bit different and contains more plugins as "pure" SDK I've used before, so might be there is more chances to hit the race conditions (more services, more threads, different activation sequence). As said, I can't reproduce, but also last time I've seen Eclipse failing to startup that badly it was (co-incident?) related to the startup changes related to the Resources plugin initialization, see bug 491850 comment 1 for the root cause.
Agreed, this is weird and I will investigate further next week.
I've debugged it a bit more. AFAICT java.lang.IllegalStateException: No instance data can be specified. at org.eclipse.core.internal.runtime.DataArea.assertLocationInitialized(DataArea.java:50) means that the service tracker instanceLocationTracker (in org.eclipse.core.internal.runtime.Activator.getInstanceLocation()) can't find the Location service of type "osgi.instance.area" (and not "osgi.configuration.area" as I said in comment #2). So the service is not registered. This can be the case for two reasons: - either SystemBundleActivator#start(BundleContext) did not reach org.eclipse.osgi.internal.framework.SystemBundleActivator.registerLocations(BundleContext, EquinoxLocations) yet - or org.eclipse.osgi.internal.location.EquinoxLocations.getInstanceLocation() returned <null> during SystemBundleActivator.registerLocations(BundleContext, EquinoxLocations). This seems to be posssible if the property "osgi.instance.area" is set to "@none". I can't see how a job can run before the end of SystemBundleActivator#start(BundleContext) (or even for any other bundle to be started before it ends) but I don't know how "osgi.instance.area" can be reasonably set to @none. Adding Ed and Eike, maybe they've already seen such a thing with Oomph. If Ed and Eike can't help, and if Sergey thinks that the issue could come from path for bug 501997, then let's revert it.
Do you have the full log still? Something more strange is going on than what is suggested so far. This services (org.eclipse.osgi.service.datalocation.Location) are registered by the Equinox framework when the framework is initializing in the method: org.eclipse.osgi.internal.framework.SystemBundleActivator.registerLocations(BundleContext, EquinoxLocations) Ultimately this registers as a service the following final field for the instance location: org.eclipse.osgi.internal.location.EquinoxLocations.instanceLocation One possibility is that something else occurred which caused the framework to shutdown quickly (causing the Location service to be unregistered) while this other thread was trying to get the service. This could be because of an error, or in some cases, when you are updating either the launcher or the framework (or one of its fragments) there is a stage upon restart where the framework is restarted multiple times before a steady state is reached. I suspect this stage of quick restarts is the likely cause of the issue. To reproduce you would have to go back to the I-Build you started with and update to the latest I-Build. But even then the timing will likely be difficult to reproduce.
(In reply to Thomas Watson from comment #7) > One possibility is that something else occurred which caused the framework > to shutdown quickly (causing the Location service to be unregistered) while > this other thread was trying to get the service. This could be because of > an error, or in some cases, when you are updating either the launcher or the > framework (or one of its fragments) there is a stage upon restart where the > framework is restarted multiple times before a steady state is reached. I > suspect this stage of quick restarts is the likely cause of the issue. To > reproduce you would have to go back to the I-Build you started with and > update to the latest I-Build. But even then the timing will likely be > difficult to reproduce. I think this could be really due the updates Oomph automatically installed. Also see bug 507812, where the NPE could only happen if the platform was shutdown during startup, so that the contentTracker was set to null via org.eclipse.core.internal.runtime.InternalPlatform.closeOSGITrackers(). Interestingly, the stack trace from bug 507812 started also in the new job from ResourcesPlugin. Can it be, that decoupling the ResourcesPlugin init code from startup allows now "faster" shutdown, so that the startup code which previously executed *before* the shutdown (in same thread with caller), now executed *during* shutdown, and so causing those NPE's?
I've just saw this error on gerrit startup, where the tests were failed due Eclipse unable to startup, see https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11224/ https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11224/artifact/tests/org.eclipse.ui.monitoring.tests/target/work/data/.metadata/.log !SESSION 2016-11-20 12:06:11.528 ----------------------------------------------- eclipse.buildId=unknown java.version=1.8.0_102 java.vendor=Oracle Corporation BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US Framework arguments: -application org.eclipse.tycho.surefire.osgibooter.uitest -testproperties /opt/public/hipp/homes/genie.platform/workspace/eclipse.platform.ui-Gerrit/tests/org.eclipse.ui.monitoring.tests/target/surefire.properties Command-line arguments: -data /opt/public/hipp/homes/genie.platform/workspace/eclipse.platform.ui-Gerrit/tests/org.eclipse.ui.monitoring.tests/target/work/data -application org.eclipse.tycho.surefire.osgibooter.uitest -testproperties /opt/public/hipp/homes/genie.platform/workspace/eclipse.platform.ui-Gerrit/tests/org.eclipse.ui.monitoring.tests/target/surefire.properties !ENTRY org.eclipse.core.jobs 4 2 2016-11-20 12:06:17.348 !MESSAGE An internal error occurred during: "Initializing workspace.". !STACK 0 java.lang.IllegalStateException: No instance data can be specified. at org.eclipse.core.internal.runtime.DataArea.assertLocationInitialized(DataArea.java:50) at org.eclipse.core.internal.runtime.DataArea.getStateLocation(DataArea.java:124) at org.eclipse.core.internal.runtime.InternalPlatform.getStateLocation(InternalPlatform.java:539) at org.eclipse.core.runtime.Plugin.getStateLocation(Plugin.java:289) at org.eclipse.core.internal.resources.LocalMetaArea.<init>(LocalMetaArea.java:59) at org.eclipse.core.resources.ResourcesPlugin$1.run(ResourcesPlugin.java:513) at org.eclipse.core.runtime.jobs.Job$2.run(Job.java:182) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56) !ENTRY org.eclipse.osgi 4 0 2016-11-20 12:06:18.018 !MESSAGE Application error !STACK 1 java.lang.ExceptionInInitializerError at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getURIConverter(ResourceSetImpl.java:499) at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(ResourceSetImpl.java:369) at org.eclipse.e4.ui.internal.workbench.ResourceHandler.getResource(ResourceHandler.java:286) at org.eclipse.e4.ui.internal.workbench.ResourceHandler.loadResource(ResourceHandler.java:262) at org.eclipse.e4.ui.internal.workbench.ResourceHandler.loadMostRecentModel(ResourceHandler.java:181) at org.eclipse.e4.ui.internal.workbench.swt.E4Application.loadApplicationModel(E4Application.java:378) at org.eclipse.e4.ui.internal.workbench.swt.E4Application.createE4Workbench(E4Application.java:253) at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:632) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:610) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:138) at org.eclipse.tycho.surefire.osgibooter.UITestApplication.runApplication(UITestApplication.java:31) at org.eclipse.tycho.surefire.osgibooter.AbstractUITestApplication.run(AbstractUITestApplication.java:120) at org.eclipse.tycho.surefire.osgibooter.UITestApplication.start(UITestApplication.java:37) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610) at org.eclipse.equinox.launcher.Main.run(Main.java:1519) at org.eclipse.equinox.launcher.Main.main(Main.java:1492) Caused by: java.lang.IllegalStateException: Workspace is closed. at org.eclipse.core.resources.ResourcesPlugin.getWorkspace(ResourcesPlugin.java:426) at org.eclipse.emf.ecore.plugin.EcorePlugin.getWorkspaceRoot(EcorePlugin.java:1131) at org.eclipse.emf.ecore.resource.impl.ExtensibleURIConverterImpl.<clinit>(ExtensibleURIConverterImpl.java:393) ... 28 more
And now I've seen another gerrit build fail due to the startup crash for org.eclipse.ui.tests.navigator test suite, see https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11235/ https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11235/artifact/tests/org.eclipse.ui.tests.navigator/target/work/data/.metadata/.log !SESSION 2016-11-21 07:41:26.484 ----------------------------------------------- eclipse.buildId=unknown java.version=1.8.0_102 java.vendor=Oracle Corporation BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US Framework arguments: -application org.eclipse.tycho.surefire.osgibooter.uitest -testproperties /opt/public/hipp/homes/genie.platform/workspace/eclipse.platform.ui-Gerrit/tests/org.eclipse.ui.tests.navigator/target/surefire.properties Command-line arguments: -data /opt/public/hipp/homes/genie.platform/workspace/eclipse.platform.ui-Gerrit/tests/org.eclipse.ui.tests.navigator/target/work/data -application org.eclipse.tycho.surefire.osgibooter.uitest -testproperties /opt/public/hipp/homes/genie.platform/workspace/eclipse.platform.ui-Gerrit/tests/org.eclipse.ui.tests.navigator/target/surefire.properties !ENTRY org.eclipse.core.jobs 4 2 2016-11-21 07:41:32.052 !MESSAGE An internal error occurred during: "Initializing workspace.". !STACK 0 java.lang.IllegalStateException: No instance data can be specified. at org.eclipse.core.internal.runtime.DataArea.assertLocationInitialized(DataArea.java:50) at org.eclipse.core.internal.runtime.DataArea.getStateLocation(DataArea.java:124) at org.eclipse.core.internal.runtime.InternalPlatform.getStateLocation(InternalPlatform.java:539) at org.eclipse.core.runtime.Plugin.getStateLocation(Plugin.java:289) at org.eclipse.core.internal.resources.LocalMetaArea.<init>(LocalMetaArea.java:59) at org.eclipse.core.resources.ResourcesPlugin$1.run(ResourcesPlugin.java:513) at org.eclipse.core.runtime.jobs.Job$2.run(Job.java:182) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56) !ENTRY org.eclipse.osgi 4 0 2016-11-21 07:41:32.802 !MESSAGE Application error !STACK 1 java.lang.ExceptionInInitializerError at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getURIConverter(ResourceSetImpl.java:499) at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.getResource(ResourceSetImpl.java:369) at org.eclipse.e4.ui.internal.workbench.ResourceHandler.getResource(ResourceHandler.java:286) at org.eclipse.e4.ui.internal.workbench.ResourceHandler.loadResource(ResourceHandler.java:262) at org.eclipse.e4.ui.internal.workbench.ResourceHandler.loadMostRecentModel(ResourceHandler.java:181) at org.eclipse.e4.ui.internal.workbench.swt.E4Application.loadApplicationModel(E4Application.java:378) at org.eclipse.e4.ui.internal.workbench.swt.E4Application.createE4Workbench(E4Application.java:253) at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:632) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:610) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:138) at org.eclipse.tycho.surefire.osgibooter.UITestApplication.runApplication(UITestApplication.java:31) at org.eclipse.tycho.surefire.osgibooter.AbstractUITestApplication.run(AbstractUITestApplication.java:120) at org.eclipse.tycho.surefire.osgibooter.UITestApplication.start(UITestApplication.java:37) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610) at org.eclipse.equinox.launcher.Main.run(Main.java:1519) at org.eclipse.equinox.launcher.Main.main(Main.java:1492) Caused by: java.lang.IllegalStateException: Workspace is closed. at org.eclipse.core.resources.ResourcesPlugin.getWorkspace(ResourcesPlugin.java:426) at org.eclipse.emf.ecore.plugin.EcorePlugin.getWorkspaceRoot(EcorePlugin.java:1131) at org.eclipse.emf.ecore.resource.impl.ExtensibleURIConverterImpl.<clinit>(ExtensibleURIConverterImpl.java:393) ... 28 more
(In reply to Thomas Watson from comment #7) > One possibility is that something else occurred which caused the framework > to shutdown quickly (causing the Location service to be unregistered) while > this other thread was trying to get the service. This could be because of > an error, or in some cases, when you are updating either the launcher or the > framework (or one of its fragments) there is a stage upon restart where the > framework is restarted multiple times before a steady state is reached. My first idea was that this was related to the Oomph setup tasks updating the build, but looking on the Gerrit failures we see now, where the "pure" SDK (with test plugins) is failing to start, I wonder if there is a different issue. What if the resources job code runs initialization code now *before* the resources bundle is "officially" activated - because the equinox is in the middle of post-processing of the bundle activation, so that the bundle state as seen from other threads is not "activated" yet, or some listeners aren't done yet with event processing, etc? @All: should we revert the original change and see if we will still get "unmotivated" Gerrit startup issues?
Adding Platform UI to the list, since this affects whole workbench startup.
This last stack trace can't really be Oomph related because I'm really quite sure the platform build doesn't include anything Oomph related. Certainly EMF expects to be able to get the workspace if the resources bundle has been deployed in the runtime: public static IWorkspaceRoot getWorkspaceRoot() { if (workspaceRoot == null && IS_RESOURCES_BUNDLE_AVAILABLE && System.getProperty("org.eclipse.emf.ecore.plugin.EcorePlugin.doNotLoadResourcesPlugin") == null) { workspaceRoot = ResourcesPlugin.getWorkspace().getRoot(); } return workspaceRoot; } Deferring the workspace creation to be later in the startup cycle (i.e., such that it is not yet available during IDEApplication.start) is not generally going to be friendly with any EMF-based application. This latest stack trace looks to be on the main thread, isn't it?
(In reply to Andrey Loskutov from comment #11) > What if the resources job code runs initialization code now *before* the > resources bundle is "officially" activated - because the equinox is in the > middle of post-processing of the bundle activation, so that the bundle state > as seen from other threads is not "activated" yet, or some listeners aren't > done yet with event processing, etc? I'm wrong of course. *Before the change* resources init happened while the plugin was not active yet, but now it happens at any other point in time *while or after* activation.
(In reply to Ed Merks from comment #13) > This last stack trace can't really be Oomph related because I'm really quite > sure the platform build doesn't include anything Oomph related. Sure. > Deferring the workspace creation to be later in the startup cycle (i.e., > such that it is not yet available during IDEApplication.start) is not > generally going to be friendly with any EMF-based application. > > This latest stack trace looks to be on the main thread, isn't it? Yes, but the first error happens in the new job initializing resources, my assumption is that the second one is just the side effect of the first.
My recommendation is to keep the patch in place for now but to add instrumentation to collect additional information needed to fully understand the problem.
(In reply to Sergey Prigogin from comment #16) > My recommendation is to keep the patch in place for now but to add > instrumentation to collect additional information needed to fully understand > the problem. Do you have something specific in mind? Or do you mean try/catch and a post-mortem diagnostic of the platfrom/metadata state in the catch block of the resources initializer job?
(In reply to Andrey Loskutov from comment #17) > Do you have something specific in mind? Or do you mean try/catch and a > post-mortem diagnostic of the platfrom/metadata state in the catch block of > the resources initializer job? Yes, with capturing stacks of all threads. If that turns out to be insufficient, additional information can be captured by temporarily enabling some tracing.
When I tried provisioning an Oxygen committers package with Oomph, I can't start the IDE with a fresh new workspace because of this stack trace: !ENTRY org.eclipse.osgi 4 0 2016-11-22 08:52:06.478 !MESSAGE Application error !STACK 1 java.lang.IllegalStateException: No instance data can be specified. at org.eclipse.core.internal.runtime.DataArea.assertLocationInitialized(DataArea.java:50) at org.eclipse.core.internal.runtime.DataArea.getStateLocation(DataArea.java:129) at org.eclipse.core.internal.preferences.InstancePreferences.getBaseLocation(InstancePreferences.java:44) at org.eclipse.core.internal.preferences.InstancePreferences.initializeChildren(InstancePreferences.java:209) at org.eclipse.core.internal.preferences.InstancePreferences.<init>(InstancePreferences.java:59) at org.eclipse.core.internal.preferences.InstancePreferences.internalCreate(InstancePreferences.java:220) at org.eclipse.core.internal.preferences.EclipsePreferences.create(EclipsePreferences.java:352) at org.eclipse.core.internal.preferences.EclipsePreferences.create(EclipsePreferences.java:340) at org.eclipse.core.internal.preferences.PreferencesService.createNode(PreferencesService.java:392) at org.eclipse.core.internal.preferences.RootPreferences.getChild(RootPreferences.java:60) at org.eclipse.core.internal.preferences.RootPreferences.getNode(RootPreferences.java:95) at org.eclipse.core.internal.preferences.RootPreferences.node(RootPreferences.java:84) at org.eclipse.core.internal.preferences.AbstractScope.getNode(AbstractScope.java:38) at org.eclipse.core.runtime.preferences.InstanceScope.getNode(InstanceScope.java:77) at org.eclipse.ui.preferences.ScopedPreferenceStore.getStorePreferences(ScopedPreferenceStore.java:228) at org.eclipse.ui.preferences.ScopedPreferenceStore.<init>(ScopedPreferenceStore.java:132) at org.eclipse.ui.plugin.AbstractUIPlugin.getPreferenceStore(AbstractUIPlugin.java:277) at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:624) at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:336) at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:610) at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:148) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:138) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:673) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:610) at org.eclipse.equinox.launcher.Main.run(Main.java:1519) It seems clear to me that we can't (shouldn't) change the life cycle for when the instance state location is available to be later than EclipseAppLauncher.start.
New Gerrit change created: https://git.eclipse.org/r/85471
I've added code to capture stacks of all threads when an exception occurs during workspace initialization (as Sergey mentionned in comment 18). The capturing/logging code is largely copied from the UI freeze monitoring plugin.
Gerrit change https://git.eclipse.org/r/85471 was merged to [master]. Commit: http://git.eclipse.org/c/platform/eclipse.platform.resources.git/commit/?id=7a4a17b7782067546a73c76cb857406cca44603b
A lot of Gerrit validations job are still failing with an error that seems related: * https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11309/artifact/tests/org.eclipse.ui.tests/target/work/data/.metadata/.log * https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11308/artifact/tests/org.eclipse.e4.ui.bindings.tests/target/work/data/.metadata/.log * https://hudson.eclipse.org/platform/job/eclipse.platform.ui-Gerrit/11307/artifact/tests/org.eclipse.ui.tests.forms/target/work/data/.metadata/.log * ...
Created attachment 265639 [details] log with thread dump from ui tests
Created attachment 265640 [details] log with thread dump from binding tests Thanks Mickael, we have now two nearly identical logs with thread dumps, but honestly speaking, I still have no clue where it goes wrong. I'm currently trying to meditate over the stacks :-)
What strikes me in both attachements is this part: !SUBENTRY 1 org.eclipse.core.resources 1 0 2016-11-29 07:35:07.192 !MESSAGE Thread ''Signal Dispatcher'' tid=4 (RUNNABLE) !STACK 0 java.lang.Exception: Stack Trace and the fact that there is no stack and that it's currently running. Maybe it's because the jvm is already terminating? Otherwise, I don't see anything suspect (esp. nothing that looks like what Thomas described in comment #7). I'll also try to meditate over the stacks, maybe it helps :D
While I can't reproduce, I share my debugging observations. If I put the breakpoint at org.eclipse.core.internal.runtime.DataArea.assertLocationInitialized() I see that multiple threads can enter the NOT thread-safe DataArea code (and it was before the main thread only). The org.eclipse.core.internal.runtime.Activator.getInstanceLocation() used by DataArea is also NOT thread safe. Also the DataArea instance used by multiple threads seem to be same, but this is also not guaranteed (see MetaDataKeeper.getMetaArea() which is again NOT multi-thread safe). So I wonder why DataArea, MetaDataKeeper and Activator.getInstanceLocation() aren't thread safe? This current bug is most likely caused by the bug in the the org.eclipse.core.internal.runtime.Activator code below, which is revealed by the commit 8d90fb030df310a974aae5d27d0bc2610c1a14ad: public Location getInstanceLocation() { if (instanceLocationTracker == null) { Filter filter = null; try { filter = bundleContext.createFilter(Location.INSTANCE_FILTER); } catch (InvalidSyntaxException e) { // ignore this. It should never happen as we have tested the above format. } instanceLocationTracker = new ServiceTracker<>(bundleContext, filter, null); instanceLocationTracker.open(); } return instanceLocationTracker.getService(); } What happens now is following: Activator.getInstanceLocation() is called from two threads - from "main" and the job triggered by the resources plugin startup (before it was "main" only). Because the method is not synchronized following happens: thread #1 enters the if() block and initializes instanceLocationTracker field with new object. AT SAME TIME, thread #2 checks if(instanceLocationTracker == null) condition and sees instanceLocationTracker value set by thread #1. Thread #2 is happy and calls instanceLocationTracker.getService() on a tracker that is not opened yet. Bingo! I have a patch.
New Gerrit change created: https://git.eclipse.org/r/85950
(In reply to Eclipse Genie from comment #28) > New Gerrit change created: https://git.eclipse.org/r/85950 Now that I know the root cause, I know how to reproduce: disable ALL breakpoints except one at line 149 in org.eclipse.core.internal.runtime.Activator.getInstanceLocation() -> instanceLocationTracker.open(); and try to start your Eclipse :-) Please note that you probably will need to disable org.eclipse.ui.trace plugin startup because it locks org.eclipse.core.internal.preferences.RootPreferences while initialization and so the "main" thread can't do bad things :-)
Gerrit change https://git.eclipse.org/r/85950 was merged to [master]. Commit: http://git.eclipse.org/c/equinox/rt.equinox.bundles.git/commit/?id=1d3b01d9c5aed305b8280d281dc165f1b3cd0cb1
(In reply to Andrey Loskutov from comment #27) > I have a patch. Thanks for tracking this down! I reworked the patch a bit. To avoid using trackers before they are opened I moved the construction of them to be in the start() method instead of lazily. There is very little gain in doing this lazily.
(In reply to Thomas Watson from comment #31) > (In reply to Andrey Loskutov from comment #27) > > I have a patch. > > Thanks for tracking this down! I reworked the patch a bit. To avoid using > trackers before they are opened I moved the construction of them to be in > the start() method instead of lazily. There is very little gain in doing > this lazily. Thanks Thomas for fast review. Beside the merged equinox patch, we should verify that other code used by resources plugin at startup is multi thread safe. @Mickael, Sergey: we schould track it under dedicated bug for resources component, since the multi threaded init is coming from there. Do you agree with that?
(In reply to Andrey Loskutov from comment #32) > @Mickael, Sergey: we schould track it under dedicated bug for resources > component, since the multi threaded init is coming from there. Do you agree > with that? Yes.
(In reply to Andrey Loskutov from comment #32) > @Mickael, Sergey: we schould track it under dedicated bug for resources > component, since the multi threaded init is coming from there. Do you agree > with that? Of course. I've created bug 508419 for that purpose.
New Gerrit change created: https://git.eclipse.org/r/93665
Gerrit change https://git.eclipse.org/r/93665 was merged to [master]. Commit: http://git.eclipse.org/c/platform/eclipse.platform.resources.git/commit/?id=60694beddcd0c8e2a84c1553e91318ff40a708c1
