Bug 513188 - [WorkingSets] projects disappear from working sets after restart
Summary: [WorkingSets] projects disappear from working sets after restart
Status: REOPENED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.6   Edit
Hardware: PC Linux
: P3 major with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: Platform-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-03-06 12:34 EST by Andrei Pozolotin CLA
Modified: 2021-12-06 11:36 EST (History)
5 users (show)

See Also:


Attachments
step #1 - new xml is empty after old xml is removed (254 bytes, text/xml)
2017-03-07 13:13 EST, Andrei Pozolotin CLA
no flags Details
step #2 - user working sets with some projects after eclipse exit (1.39 KB, text/xml)
2017-03-07 13:14 EST, Andrei Pozolotin CLA
no flags Details
step #3 - xml after eclipse restart, and after eclipse done re-parse of xml (592 bytes, text/xml)
2017-03-07 13:16 EST, Andrei Pozolotin CLA
no flags Details
step #3 - working sets are still OK in the UI, while persisted xml does not reflect that (12.04 KB, image/png)
2017-03-07 13:18 EST, Andrei Pozolotin CLA
no flags Details
step #3 - one more restart, now both UI and xml are in sync, "lost proj" are in "Other Projects" (6.92 KB, image/png)
2017-03-07 13:42 EST, Andrei Pozolotin CLA
no flags Details
typical eclipse log during steps #1...3: nothing reported about xml re-parse, while xml is changed (20.27 KB, text/x-log)
2017-03-07 14:18 EST, Andrei Pozolotin CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrei Pozolotin CLA 2017-03-06 12:34:05 EST
--------------
system:

uname -a
Linux ecli-main-4-6 4.9.11-1-ARCH #1 SMP PREEMPT Sun Feb 19 13:45:52 UTC 2017 x86_64 GNU/Linux

java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

Eclipse Platform
Version: Neon.2 (4.6.2)
Build id: M20161124-1400

--------------
problem:

happens with 100% consistency:

0) precondition: start with empty workingsets.xml:
remove .../.metadata/.plugins/org.eclipse.ui.workbench/workingsets.xml

1) start eclipse #1, create some working sets, assign projects
verify workingsets.xml contains newly created sets with member projects

2) exit and start eclipse again #2
a) observe in UI that working sets all look same as before in #1
b) observe that workingsets.xml is re-parsed some sets LOST some projects

3) exit and start eclipse again #3
a) observe in UI that some sets LOST some projects from #1
b) observe that corrupt workingsets.xml is consistent with what is shown in UI

--------------
solution:

a) have a startup script which re-creates original workingsets.xml from the backup
b) have a shutdown script which makes a backup of the workingsets.xml for next start
Comment 1 Andrey Loskutov CLA 2017-03-07 07:19:40 EST
We use working sets to organize ~180 project but did not observed this problem yet and so never tried to modify the file manually.

Do you see any errors in Eclipse log? 
How many projects do you have? 
Do you use Oomph (because it has a "Dynamic" working set management)?
The title says "corrupt". Is this really the case? Is the xml structure corrupt, or just some data is missing (as if someone would remove some projects)?
Comment 2 Andrei Pozolotin CLA 2017-03-07 12:50:00 EST
(In reply to Andrey Loskutov from comment #1)
> We use working sets to organize ~180 project but did not observed this
> problem yet and so never tried to modify the file manually.
we have around 100 projects. we have lots of plugins.
we did not have problem for few month after migration 4.5 -> 4.6
then this problem appeared, we can not trace it back.

> Do you see any errors in Eclipse log? 
yes, from various plugins but nothing which seems relevant 
to the workspaces manager.

> How many projects do you have? 
we have around 100 projects. 

> Do you use Oomph (because it has a "Dynamic" working set management)?
we have NO dynamic projects.

> The title says "corrupt". Is this really the case? Is the xml structure
> corrupt, or just some data is missing (as if someone would remove some
> projects)?
ok, "corrupt" is wrong term here. xml file itself is fine (and can be validated), but the projects are getting removed from custom user working sets and are moved into "Other Projects" catch-all eclipse working set. Custom user working sets are still there, but empty after re-parse on start up of eclipse. 

I will reproduce steps 1 ... 4 above and will provide workingsets.xml samples
Comment 3 Andrei Pozolotin CLA 2017-03-07 13:13:21 EST
Created attachment 267148 [details]
step #1 - new xml is empty after old xml is removed
Comment 4 Andrei Pozolotin CLA 2017-03-07 13:14:37 EST
Created attachment 267149 [details]
step #2 - user working sets with some projects after eclipse exit
Comment 5 Andrei Pozolotin CLA 2017-03-07 13:16:12 EST
Created attachment 267150 [details]
step #3 - xml after eclipse restart, and after eclipse done re-parse of xml
Comment 6 Andrei Pozolotin CLA 2017-03-07 13:18:14 EST
Created attachment 267151 [details]
step #3 - working sets are still OK in the UI, while persisted xml does not reflect that
Comment 7 Andrey Loskutov CLA 2017-03-07 13:21:57 EST
Do the closed projects from the screenshot still exist?
Comment 8 Andrei Pozolotin CLA 2017-03-07 13:38:03 EST
(In reply to Andrey Loskutov from comment #7)
> Do the closed projects from the screenshot still exist?

yes, they appear in the "Other Projects" default eclipse working set
Comment 9 Andrei Pozolotin CLA 2017-03-07 13:42:07 EST
Created attachment 267152 [details]
step #3 - one more restart, now both UI and xml are in sync, "lost proj" are in "Other Projects"
Comment 10 Andrey Loskutov CLA 2017-03-07 13:49:07 EST
(In reply to Andrei Pozolotin from comment #8)
> (In reply to Andrey Loskutov from comment #7)
> > Do the closed projects from the screenshot still exist?
> 
> yes, they appear in the "Other Projects" default eclipse working set

I meant, do they still *physically* exist on the disk?
Comment 11 Andrei Pozolotin CLA 2017-03-07 14:10:17 EST
> I meant, do they still *physically* exist on the disk?

1) yes, they are different java, scala, php, etc. projects, which build and behave normally.

2) your suspicion about "existence" brings the following point:

could it be that during v 4.5 -> 4.6 change workspace manager can not see custom
workspace paths specified by "osgi.instance.area"?

we use the following eclipse invocation entries:

### eclipse command
java
-Xverify:none
-XX:+UnlockCommercialFeatures
-XX:+FlightRecorder
-server
-XX:+UseG1GC
-XX:+DoEscapeAnalysis
-XX:+UseCompressedOops
-XX:+UseCompressedClassPointers
-XX:+HeapDumpOnOutOfMemoryError
-XX:ThreadStackSize=1m
-XX:InitialHeapSize=4000m
-XX:MaxHeapSize=4000m
-XX:MetaspaceSize=700m
-XX:MaxMetaspaceSize=700m
-XX:CompressedClassSpaceSize=100m
-XX:InitialCodeCacheSize=300m
-XX:ReservedCodeCacheSize=300m
-Dosgi.configuration.area=/home/work/space/main-4.6.0/conf
-Dosgi.instance.area=/home/work/space/main-4.6.0/data
-Dosgi.user.area=/home/user0/Eclipse
-jar
/opt/eclipse/plugins/org.eclipse.equinox.launcher_1.3.201.v20161025-1711.jar
-clean
### eclipse command
Comment 12 Andrey Loskutov CLA 2017-03-07 14:16:36 EST
Do you see anything in common between "disappearing" projects, or they disappear absolutely random?
Comment 13 Andrei Pozolotin CLA 2017-03-07 14:18:43 EST
Created attachment 267154 [details]
typical eclipse log during steps #1...3: nothing reported about xml re-parse, while xml is changed
Comment 14 Andrey Loskutov CLA 2017-03-07 14:22:22 EST
(In reply to Andrei Pozolotin from comment #13)
> Created attachment 267154 [details]
> typical eclipse log during steps #1...3: nothing reported about xml
> re-parse, while xml is changed

Is there any reason why you start with -clean argument? Is this always the case?
Comment 15 Andrei Pozolotin CLA 2017-03-07 14:27:31 EST
(In reply to Andrey Loskutov from comment #12)
> Do you see anything in common between "disappearing" projects, or they
> disappear absolutely random?

1) projects disappear mostly at random

2) projects differ a little depending on "<item/>" type:

a) "elementID-based" appear to be "more stable", takes few restarts to break them
<item elementID="=sjs_sri-web-template" factoryID="org.eclipse.jdt.ui.PersistableJavaElementFactory"/>

b) "path-based" disappear pretty much on first restart:
<item factoryID="org.eclipse.ui.internal.model.ResourceFactory" path="/sjs_libanius-scalajs-react-app-js" type="4"/>

3) given enough eclipse restarts, all project end up in the "Other Projects" working set
Comment 16 Andrey Loskutov CLA 2017-03-07 14:33:44 EST
Cool or crazy. Can you please try to open your "full" workspace (with all working sets in the good state) with the plain SDK (I see you use lot of 3rd party plugins) , and check if SDK would also corrupt your working sets?

This is so crazy it must be some plugin modifying working sets. I never ever lost working sets in 10 years or so, and I use them all the time.
Comment 17 Andrei Pozolotin CLA 2017-03-07 14:45:16 EST
(In reply to Andrey Loskutov from comment #14)
> Is there any reason why you start with -clean argument? Is this always the
> case?

1) we used "-clean" for years. in our experience, it helps with eclipse stability and performance

2) I verified that after removal of "-clean" the workspaces.xml problem behavior does not change
Comment 18 Andrei Pozolotin CLA 2017-03-07 16:51:27 EST
(In reply to Andrey Loskutov from comment #16)
> Cool or crazy.

1) I verified that the issue is GONE when switching to plugin-free eclipse

2) but, we have some 50 top level plugin feature.groups (by vendor), 100s of features and 1000s of bundles in the installation. short of doing manual binary search, what is the way to find the offending plugin/bundle? 

3) also additional problem is that same plugin profile works ok in v 4.5
Comment 19 Andrey Loskutov CLA 2017-03-07 17:05:47 EST
(In reply to Andrei Pozolotin from comment #18)
> (In reply to Andrey Loskutov from comment #16)
> > Cool or crazy.
> 
> 1) I verified that the issue is GONE when switching to plugin-free eclipse

Thanks

> 2) but, we have some 50 top level plugin feature.groups (by vendor), 100s of
> features and 1000s of bundles in the installation. short of doing manual
> binary search, what is the way to find the offending plugin/bundle? 

I feel your pain, but this is life. I have no silver bullet for you, except you will try with debugger.

Germans would say: "da muss du durch" ... 

> 3) also additional problem is that same plugin profile works ok in v 4.5

One of the 3rd party plugins could be not properly working with 4.6?

If you can debug your install, you can check who persists the sets. I will close this issue now, but feel free to reopen if you can pinpoint the problem with any of eclipse.org based plugins as root cause.
Comment 20 Andrei Pozolotin CLA 2017-03-07 20:09:20 EST
Andrey: thank you very much for your time.
Comment 21 Andrey Loskutov CLA 2017-03-08 00:33:11 EST
(In reply to Andrei Pozolotin from comment #20)
> Andrey: thank you very much for your time.

You are welcome. Please if yoi will find the root cause, post here a short update.
Comment 22 jim jonah CLA 2018-11-16 10:08:44 EST
I ran into a similar problem.

If a third party plugin is set to autostart and that plugin happens to have something that implements (in my case) org.eclipse.debug.ui.ILaunchShortcut, then the plug gets loaded BEFORE org.eclipse.IDE does. That seems to trigger this code to be run:

private BreakpointOrganizerManager() {
        loadOrganizers();
        // force the working set organizers to initialize their listeners
        start("org.eclipse.debug.ui.workingSetOrganizer"); //$NON-NLS-1$
        start("org.eclipse.debug.ui.breakpointWorkingSetOrganizer"); //$NON-NLS-1$

which cases the workingsets.xml to be read and then persisted.

It's during the saving that things go wrong. Since IDE hasn't be loaded yet this code has not been run yet:

IAdapterManager manager = Platform.getAdapterManager();
IAdapterFactory factory = new WorkbenchAdapterFactory();
manager.registerAdapters(factory, IWorkspace.class);
manager.registerAdapters(factory, IWorkspaceRoot.class);
manager.registerAdapters(factory, IProject.class);
manager.registerAdapters(factory, IFolder.class);
manager.registerAdapters(factory, IFile.class);
manager.registerAdapters(factory, IMarker.class);

// properties adapters
IAdapterFactory paFactory = new StandardPropertiesAdapterFactory();
manager.registerAdapters(paFactory, IWorkspace.class);
manager.registerAdapters(paFactory, IWorkspaceRoot.class);
manager.registerAdapters(paFactory, IProject.class);
manager.registerAdapters(paFactory, IFolder.class);
manager.registerAdapters(paFactory, IFile.class);
manager.registerAdapters(paFactory, IMarker.class);

Which means that the persistence of the memento doesn't now how to save the elements of the working set (In my case they were IFiles).

The end result is the workingset.xml on disk is missing the <items> and the NEXT time eclipse is started they will no longer be loaded.
Comment 23 Andrey Loskutov CLA 2018-11-16 10:17:17 EST
(In reply to jim jonah from comment #22)
> I ran into a similar problem.
> 
> If a third party plugin is set to autostart and that plugin happens to have
> something that implements (in my case) org.eclipse.debug.ui.ILaunchShortcut,
> then the plug gets loaded BEFORE org.eclipse.IDE does. That seems to trigger
> this code to be run:
> 
> private BreakpointOrganizerManager() {
>         loadOrganizers();
>         // force the working set organizers to initialize their listeners
>         start("org.eclipse.debug.ui.workingSetOrganizer"); //$NON-NLS-1$
>         start("org.eclipse.debug.ui.breakpointWorkingSetOrganizer");
> //$NON-NLS-1$
> 
> which cases the workingsets.xml to be read and then persisted.
> 
> It's during the saving that things go wrong. Since IDE hasn't be loaded yet
> this code has not been run yet:
> [...]
> Which means that the persistence of the memento doesn't now how to save the
> elements of the working set (In my case they were IFiles).
> 
> The end result is the workingset.xml on disk is missing the <items> and the
> NEXT time eclipse is started they will no longer be loaded.

Cool. I'm curious, how did you find this?
So the solution would be that BreakpointOrganizerManager should make sure IDE plugin is loaded first, right?
Comment 24 jim jonah CLA 2018-11-16 15:37:19 EST
breakpoints in:
AdapterManager.registerAdapters
WorkingSetManager.saveWorkingSetState(IMemento memento)
WorkingSet.saveState(Imemento memento) (specifically the while loop at IPersitableElement persistable = ...

If the AdapterManager isn't fully loaded then persistable will be null, which is why the <item> tag doesn't get written for a workingset.

The fix? I don't know the internals of eclipse well enough to dictate that, however, it would seem that having the AdapterManager register the adapters I listed before the WorkSetManager runs is key.

For us I reworked the plugin that implements org.eclipse.debug.ui.ILaunchShortcut so that I didn't need to have autostart set to true. However, that won't help other 3rd party plugins if they are doing something similar
Comment 25 Eclipse Genie CLA 2020-11-06 03:56:18 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 26 John Smith CLA 2021-12-06 09:27:28 EST
This bug is still in existence in the latest Eclipse release (4.21) and I still have it. Please kindly reopen the issue.
Comment 27 John Smith CLA 2021-12-06 09:28:24 EST
It can be reproduced as described originally.
Comment 28 Andrey Loskutov CLA 2021-12-06 11:36:25 EST
Short summary: 

org.eclipse.ui.ide.IDE.registerAdapters() must be called before org.eclipse.debug.internal.ui.views.breakpoints.BreakpointOrganizerManager.getDefault() can be used.

Today that IDE.registerAdapters() is only called in org.eclipse.ui.internal.ide.application.IDEWorkbenchAdvisor.initialize(IWorkbenchConfigurer), and that is "too late" if some bundle calls BreakpointOrganizerManager.getDefault() before.

In that case not all adapter factories are loaded and so org.eclipse.ui.internal.WorkingSetManager.saveState() that happens early enough might not persist all working sets that were read from disk.