Bug 482735 - Repeatable workspace corruption
Summary: Repeatable workspace corruption
Status: CLOSED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 4.5.1   Edit
Hardware: PC Linux
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: Platform-Resources-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2015-11-20 19:29 EST by David M. Karr CLA
Modified: 2020-03-04 17:06 EST (History)
2 users (show)

See Also:


Attachments
diff of "bad" workspace3 and new workspace 5 (998.57 KB, patch)
2015-11-21 13:04 EST, David M. Karr CLA
no flags Details | Diff
diff of "bad" workspace3 and newly bad workspace 5 (1007.82 KB, patch)
2015-11-21 13:05 EST, David M. Karr CLA
no flags Details | Diff
Diff of bad workspace6 with new workspace7 (759.89 KB, patch)
2015-12-07 12:38 EST, David M. Karr CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description David M. Karr CLA 2015-11-20 19:29:52 EST
I'm seeing a repeatable situation that corrupts my Eclipse workspace on one of two hosts, forcing me to recreate it and reimport all of my projects.  The circumstances are as follows:

I'm working with an open-source project, hosted at https://github.com/xored/yang-ide .

I have this cloned to my CentOS7.1 laptop, running Mars.1 Eclipse.  I imported the projects to my workspace, without copying into the workspace.  This workspace is working perfectly fine.

On this laptop, I have a Ubuntu 14.04 VM.  I have Mars.1 Eclipse also installed on this, along with a couple of other applications.  I have no particular problem with any of the applications running on this VM, except for the Eclipse problem that I'm describing.

I've defined a shared folder on this VM that points to my home directory on the "host".  I've used this for copying files back and forth between the host and VM.  I've never seen any issues with files transferred back and forth.

In the workspace I created on the VM, I selected "Import Existing Projects into Workspace", and selected the path that points to the same folder on the "host" that I used to import the projects into the "host" workspace.  I did not turn on the "Copy Projects into workspace" checkbox.

After the import finished, I changed the "Target Platform" to refer to the target platform defined in the imported projects.  This fixed all the remaining relevant "red Xes" in my projects.

I use this workspace on the VM to create a test instance that I can then remotely connect to from the "host" so that I can safely step through code in the plugin without confusing my user interface, or locking it up (stepping through code while creating context menus is dangerous).

I worked with this dual-workspace setup for a couple of days, testing a fix that I made.

After a couple of days, where I would restart the VM about daily, I restarted it again, and instead of coming up normally, it showed an empty project explorer and displayed the following error in the editor view: Cannot determine URI for '/com.cisco.yangide.editor/src/com/cisco/yangide/editor/editors/YangAutoIndentStrategy.java'.

Note that I've posted about this problem at https://www.eclipse.org/forums/index.php/m/1715286/#msg_1715286 .

As I also describe in that post, ended up using this remote setup to step through the Eclipse code that shows this error message.  I found that it can't find this file because it can't find the project it's contained in.

When this error happens, I've looked carefully at the project files in the now corrupted workspace, but it seems reasonable to me.

I've now had this corruption happen three times.  In each case, I've gone through the same steps, switching to the new workspace and reimporting the projects (not copying files into the workspace).  After this last occurrence, I'm going to try creating the workspace by copying the files into the workspace.  This will be an unfortunate workaround, as it means I will have to do this again every time I changed my source in preparation for testing.  I'm only doing this to try to eliminate variables.
Comment 1 David M. Karr CLA 2015-11-21 13:04:00 EST
Created attachment 258198 [details]
diff of "bad" workspace3 and new workspace 5

I ran this diff of the old existing workspace3, after it had failed a while ago, and the new workspace5, right after creating the workspace, importing my projects, setting the target platform, and exiting Eclipse.
Comment 2 David M. Karr CLA 2015-11-21 13:05:46 EST
Created attachment 258199 [details]
diff of "bad" workspace3 and newly bad workspace 5

I ran this diff of the old existing "bad" workspace3 and the now bad workspace5 after working with workspace5 for a while, restarting Eclipse, and seeing that it had the bad symptoms, just like workspace3.
Comment 3 Brian de Alwis CLA 2015-11-24 10:20:25 EST
You didn't say what VM software you were using, but I have seen corruption across a shared-host drive with VMWare and Linux.  I ended up mounting my home using SMB and had no corruption.
Comment 4 David M. Karr CLA 2015-11-24 19:52:04 EST
I'm using VirtualBox (4.3).  I frankly find it hard to believe this is some sort of problem with shared folders.  The files that are allegedly corrupted (although I can't determine for sure whether any files are corrupted at all) are all in the local workspace.
Comment 5 Brian de Alwis CLA 2015-11-24 22:48:06 EST
(In reply to David M. Karr from comment #1)
> In each case, I've gone through the same steps, switching to the new workspace
> and reimporting the projects (not copying files into the workspace).

You reported that you were using the shared folders.  All I'm suggesting it to try mounting your projects via NFS rather than the shared folders.

Or put another way: you didn't mention seeing the same corruption in your CentOS host.  If such corruption was due to Java/Eclipse, we'd be be receiving numerous reports.  That points to your VM as the culprit.
Comment 6 David M. Karr CLA 2015-11-25 13:04:16 EST
And again, I'm not mounting "projects" remotely, I'm mounting the imported source code of the projects.  The projects themselves are all local to the workspace.  The error I'm getting is because Eclipse thinks the projects don't exist, which is represented by files on the local filesystem.
Comment 7 David M. Karr CLA 2015-12-07 12:36:44 EST
Another workspace bit the dust this morning.  It's been a while since the last corruption, so I thought perhaps this had gone away, but that's not the case.

I will repeat, I am using shared folders, but only for the source code of the project.  The workspace resides on the local virtual disk.  When I import the projects, I specify to not copy the files into the workspace.

When these corruption errors happen, it isn't the source files that disappear, it's the project data itself, which is all stored in the local workspace, not on the shared folder.  It also isn't a transient memory error, as I can restart the failed workspace or even reboot the VM, but once a workspace gets corrupted, it's corrupted.

In each of these cases, starting over means creating a new workspace, importing all the projects again, setting my target platform to the one defined in the imported project, and recreating my run configuration.

Today, after following through with these tasks, I then ran a "diff -cr" between the latest "bad" workspace and the new workspace.  I've edited the diff file somewhat, because it's pretty large.  I removed the diff reports for some files if the only differences were in the workspace path, or in the last timestamp for a file.  If it matters, I'm uploading this file to this report.
Comment 8 David M. Karr CLA 2015-12-07 12:38:03 EST
Created attachment 258487 [details]
Diff of bad workspace6 with new workspace7

This diff listing has some files removed from it, if the file with differences only showed differences in the workspace name, or in the last modified timestamp.
Comment 9 Brian de Alwis CLA 2015-12-07 14:03:22 EST
Could you try using the a developer build of Eclipse from:

    https://www.eclipse.org/downloads/index-developer.php

I remembered that I had a corrupt workspace that I kept around intending to debug, and it works fine with the latest Eclipse.

Moving to Platform/Resources.
Comment 10 David M. Karr CLA 2015-12-07 15:13:50 EST
Good idea, but it didn't make any difference.  I installed Neon M3, created a new workspace, imported all the projects, and then I switched to the most recently corrupted Mars workspace (accepting the "we're going to update this workspace") prompt), and it showed the same symptoms as I saw in Mars.  I guess I'll continue using the Neon instance and new workspace to see whether this one gets corrupted eventually.
Comment 11 David M. Karr CLA 2015-12-08 12:38:58 EST
And after barely a day using the Neon workspace, it's corrupted in the same way.

I guess what I'm going to do now is create another workspace with Neon and import the projects, and then I'm going to archive the workspace and use it until that one fails and then try to do a comparison of the corrupted workspace with the archive.
Comment 12 Brian de Alwis CLA 2015-12-08 13:09:20 EST
Could you try importing your source so that you're not using the shared folders at all?

Is there any way you could zip up the corrupt workspace?
Comment 13 David M. Karr CLA 2015-12-08 13:17:34 EST
I suppose I'll have to try copying the source in as another test.

Zipping up the workspace is easy.  Uploading a 300+mb archive is another thing.
Comment 14 David M. Karr CLA 2015-12-08 16:39:34 EST
Another relevant point I should make is that the files that I'm referencing through the shared folder are in my "git" tree of cloned projects on the host laptop.  I have the same projects imported into my Mars.1 instance on the host (also not copied into workspace), and I have never seen any corruption of the workspace on the host.  The only other difference between the host and the guest is that the host is CentOS7.1, and the guest is Ubuntu14.04.

This seems to add more to the argument that the shared folder is contributing to the corruption, but it is pretty clear that the files referenced in the shared folder are NOT corrupted.  It's the local project files on the guest that are being corrupted.  It's entirely possible some sort of communication error with the shared folder is causing Eclipse to corrupt the local project files.  Once the workspace is corrupted, it doesn't recover, but I'm able to create another workspace using the same imported files.
Comment 15 Szymon Ptaszkiewicz CLA 2016-02-24 08:46:44 EST
I completely don't understand what you are trying to do but it all sounds over complicated. Please provide a simpler explanation and steps that could be reproduced locally. Otherwise, I don't see anything that we can do here.
Comment 16 David M. Karr CLA 2016-04-21 10:21:31 EDT
Concerning "what I'm trying to do", it seems pretty straightforward to me.

I have the git repo for my Eclipse plugin on my CentOS laptop.  I run an UBuntu VM with VirtualBox.  I have a shared folder so I can reach the home directory of the main laptop from the VM.  I view and work on the application on Eclipse running on the CentOS laptop.

I run Eclipse on the VM.  I import existing projects, pointing to the git repo on the laptop.  I don't "copy into workspace", I just reference them remotely.

I then create an "Eclipse Application" run configuration on the VM, along with the "Xdebug" parameters so I can debug it remotely. I launch the run configuration.  I go back to Eclipse on the laptop and launch a debug configuration, connecting to the IP of the VM.  I exercise the plugin in the launched Eclipse on the VM, stepping through the code on the CentOS Eclipse instance.  If I bring up a menu on the VM instance, I can step through the code that constructs it without locking up the system.

The failure that happens is that sometimes when I bring up the Eclipse instance on the VM (not from the launch config), which I had imported the projects into, it comes up "brain dead", thinking I have no projects.  If I had been viewing any files from those projects the last time it was working, it shows those tabs, but with no contents but error messages.  At that point, the workspace is essentially corrupted, so I have to create a new workspace and reimport all the projects again.  This used to happen every day until I got smart(er) and started "saving the state of the VM" instead of shutting it down, so I could avoid shutting down the Eclipse instance.
Comment 17 David M. Karr CLA 2016-04-21 11:58:18 EDT
As Brian suggested in the thread I started (https://www.eclipse.org/forums/index.php?t=msg&th=1076707&goto=1729990&#msg_1729990), I've now set up the standalone instance on the VM so I can step through its startup on Eclipse on the CentOS host.  I set breakpoints in "ResourcesPlugin.start()".  I saved a VM snapshot, noting that the workspace is currently not corrupted.  When I hit the bug again, I'll save another VM snapshot and then restore that and step through the startup.

In the "working" state, when I stepped through "ResourcesPlugin.start()" and the following "Workspace.open()", I didn't really see anything significant happening.  I obviously didn't see anything that looked like a failure, but I also didn't see where it was referencing projects.  I'm not sure where to drill down here.
Comment 18 Szymon Ptaszkiewicz CLA 2016-04-21 12:10:14 EDT
(In reply to David M. Karr from comment #16)

Thanks a lot for clarification. I have some questions:

1. In comment 6 you mentioned that "projects themselves are all local to the workspace" but you also mentioned in comment 0, comment 7 and comment 16 that you "did not turn on the "Copy Projects into workspace" checkbox". Can you please explain how is it exactly configured that project is local to the workspace in your VM but is not copied to that workspace at the same time?

2. In comment 14 you mentioned that "It's the local project files on the guest that are being corrupted.". Can you please explain which files you are referring to exactly?
Comment 19 David M. Karr CLA 2016-04-21 12:24:37 EDT
(In reply to Szymon Ptaszkiewicz from comment #18)
> (In reply to David M. Karr from comment #16)
> 
> Thanks a lot for clarification. I have some questions:
> 
> 1. In comment 6 you mentioned that "projects themselves are all local to the
> workspace" but you also mentioned in comment 0, comment 7 and comment 16
> that you "did not turn on the "Copy Projects into workspace" checkbox". Can
> you please explain how is it exactly configured that project is local to the
> workspace in your VM but is not copied to that workspace at the same time?
> 
> 2. In comment 14 you mentioned that "It's the local project files on the
> guest that are being corrupted.". Can you please explain which files you are
> referring to exactly?

Some of these statements are based on my assumptions about how Eclipse works, and the evidence that I'm seeing here.

My git repo is on the CentOS host.  I work on that code there.  I've never seen any problem with the workspace on that box.

I assume that when I "import from existing projects", but I don't select the "copy into workspace" flag, although all the "code" for the project resides elsewhere, eclipse had to create SOMETHING in the workspace to represent the project, otherwise when I restarted again it would have no record of the project in the workspace.

When the bug happens, it sure looks like the projects are gone, but there's nothing wrong with the projects in the git repo, and Eclipse on the CentOS host has no trouble restarting and showign the projects.  Again, to fix this all I do at that point on the VM is create a new workspace and reimport the projects again (and set target platform, and update maven projects, and ignore maven lifecycle errors, et cetera), and they are fine, until the bug happens again.

From this experience, I conclude that the files in the git repo are never corrupted, but metadata files representing the workspace projects in the VM's workspace are.  I have no idea what files are being corrupted, but the evidence tells me that something in the Eclipse metadata on the VM thinks those projects are gone, and they are clearly fine.
Comment 20 Eclipse Genie CLA 2020-03-04 17:06:56 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug.

If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.