Bug 177148 - Project import fails on version controlled .project
Summary: Project import fails on version controlled .project
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 3.2.2   Edit
Hardware: PC Windows XP
: P3 normal with 1 vote (vote)
Target Milestone: 3.4 M1   Edit
Assignee: John Arthorne CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed
: 193405 217232 (view as bug list)
Depends on:
Blocks: 322821
  Show dependency tree
 
Reported: 2007-03-13 10:37 EDT by jason evans CLA
Modified: 2010-08-17 05:26 EDT (History)
6 users (show)

See Also:


Attachments
simple patch (see prev. comment) (1.12 KB, patch)
2007-03-29 10:46 EDT, Jacek Pospychala CLA
no flags Details | Diff
updated patch (1.78 KB, patch)
2007-06-28 09:35 EDT, Jacek Pospychala CLA
no flags Details | Diff
tests (3.47 KB, patch)
2007-06-28 09:37 EDT, Jacek Pospychala CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description jason evans CLA 2007-03-13 10:37:33 EDT
Build ID:  M20070212-1330

Steps To Reproduce:
1. Checkout .project file
2. Import existing eclipse 3.1 project
3. Checkin .project file
4. Delete project
5. Import existing java project
6. Canceling project import and refreshing project from tree will accurately reflect project files.


More information:
We are seeing occasional import failures with the following errors.  Between 4 developers it happens 50% of the time. The .project file is under source control.

"Error processing changed links in project file."
The details indicates the following lines
File /common/.project is read-only.
File /common/.project is read-only.

By checking out the .project file before import and diffing the changes we see that eclipse will only change the order of the 2 linked resources.

All the effected project files have 2 linked resources:
<linkedResources>
		<link>
			<name>Suspended Root</name>
			<type>2</type>
			<locationURI>SuspendedRootVariable</locationURI>
		</link>
		<link>
			<name>classes</name>
			<type>2</type>
			<locationURI>CLASSES/common/common</locationURI>
		</link>
Comment 1 Jacek Pospychala CLA 2007-03-14 04:05:23 EDT
Another problem caused by reversing linked resources order in .project file:

When a project has nested linked folders, like:
dirA/dirB  (both dirA and dirB are linked dirs)

In every second import link dirB is first on links list in .project file. It is created prior to dirA. Becuase dirB parent doesn't exist in that moment, it's created as a regular folder. Following that "dirA" link is not created and all it's contents are not accessible.
Comment 2 Jacek Pospychala CLA 2007-03-15 04:15:18 EDT
I think there are two issues concerned in this bug:
1. .project file is modified in the import time. It should be preserved, or a new project becomes actually a different than the one it was before import.
2. one of cases when (1.) happens is, that linked folders order in .project file is not preserved. This is what i think John Evans experiences. this is bug 177367.

I am not sure if fixing 177367 will stop Eclipse from overwriting .project.
Comment 3 John Arthorne CLA 2007-03-28 15:32:35 EDT
Jacek, can you verify if your fix for bug 177367 addresses this?
Comment 4 Jacek Pospychala CLA 2007-03-29 10:41:58 EDT
(In reply to comment #3)
> Jacek, can you verify if your fix for bug 177367 addresses this?
> 

no it does not. In this case, after project import, recreated links enforce rewriting ".project". Since links are every time in different order (HashMap), resource manager decides to really write project description but without success (file is readonly).
Simple patch is to sort links before writing.
Difficult patch is to make FileSystemResourceManager.descriptionChanged(...) more sophisticated.
Comment 5 Jacek Pospychala CLA 2007-03-29 10:46:15 EDT
Created attachment 62391 [details]
simple patch (see prev. comment)

so far I had no luck creating test cases :|
Comment 6 John Arthorne CLA 2007-03-29 11:19:14 EDT
HashMap should still iterate in a consistent order, so I don't think it will write links differently every time.  I suspect this might only happen when importing from Eclipse 3.1, since the format of the .project file changed (we introduced "locationURI" for links to non-local file systems).  We can't avoid writing the file in this case because the format changed was required.  See also bug 152591. 
Comment 7 jason evans CLA 2007-03-29 11:40:38 EDT
I would like to add that after the auto-magic upgrade of the project from 3.1 to 3.2 and checking in the upgraded 3.2 project into source control we still encounter the linked resource re-ordering issue on import. Let me reiterate that its success rate seems to be ~50%.
I would agree that the hashmap should maintain a predicatable iteration order unless it gets copies, or changed. A linkedhashmap may also be a solution here.
Comment 8 Jacek Pospychala CLA 2007-03-30 03:32:30 EDT
(In reply to comment #7)
> I would like to add that after the auto-magic upgrade of the project from 3.1
> to 3.2 and checking in the upgraded 3.2 project into source control we still
> encounter the linked resource re-ordering issue on import. Let me reiterate
> that its success rate seems to be ~50%.
> I would agree that the hashmap should maintain a predicatable iteration order
> unless it gets copies, or changed. A linkedhashmap may also be a solution here.
> 

Jason, then what is the order of links in imported .project now?
Recently i had issue that every time while importing .project, linked resources were reversed, so in 50% cases order was incorrect (but file was read/write):
A, B - order correct
B, A - bad
A, B - ok
B, A - bad
---------
50% bad

but can't reproduce it now. Does whole team use Eclipse 3.2.2? Which java do you use?
I've checked 3.1.2, 3.2, 3.2.1, 3.2.2, 3.3m4 and every one of them tries to reverse if links are 1."Suspended Root" 2."classes"
but none, when they are "classes", "Suspended Root", can you verify you've got the same with your Eclipse?
Comment 9 jason evans CLA 2007-03-30 08:55:13 EDT
Eclipse v3.2.2
Java 1.5.0_11
Sampling about 10 projects in src control most are ordered 
1)classes
2)Suspended root
Most developers are "import it and forget it" so I can not get any straight answers from the group. 
I just deleted 2 projects with the linked resources in different orders between the 2 projects and both imports failed.
a,b - fail
b,a - fail

At times during the building up the our own eclipse archive for the development team and constantly importing, deleting, and re-importing projects to identify this issue I has great runs of 'luck' where all projects imports were successfull for a string of time (hours)         

FWIW the v3.1 projects all had the LR order of
1) Classes
2) Suspended Root
Comment 10 Jacek Pospychala CLA 2007-04-11 10:12:32 EDT
I've just made following test:

HashMap<String, String> hm = new HashMap<String, String>();
		
		hm.put("Classes", "Classes");
		hm.put("classes", "classes");
		hm.put("Suspended Root", "Suspended Root");
		hm.put("suspended Root", "suspended Root");
		
		for(String v : hm.values()) {
			System.out.println(v);
		}

and it results with:

suspended Root
Classes
classes
Suspended Root

(using sun jdk 1.5)
So as you see, the issue may depend even on letter case (in fact LinkDescription's).

(In reply to comment #6): order is consistent, but depends on HashMap and LinkDescription.hashCode (in fact String.hashCode) implementations, which may vary on different machines/jdk's

i think it would be better to directly set the order that will be always the same (read: apply patch in comment #5) and don't rely on hash order.


> We can't avoid
> writing the file in this case because the format changed was required.  See
> also bug 152591. 

this is true and can't be fixed :|
Comment 11 John Arthorne CLA 2007-06-25 17:38:13 EDT
*** Bug 193405 has been marked as a duplicate of this bug. ***
Comment 12 John Arthorne CLA 2007-06-25 17:42:19 EDT
Patch looks good. Some extra things that need to be done:

 - Also ensure we write builder arguments in a consistent order (ModelObjectWriter.write(String, Map, XMLWriter).
 - Add JUnit tests.
Comment 13 Jacek Pospychala CLA 2007-06-25 17:56:47 EDT
thanks John! i'll try to update patch no later than to the end of the week.
Comment 14 Jacek Pospychala CLA 2007-06-28 09:35:27 EDT
Created attachment 72681 [details]
updated patch

updated according your comments
Comment 15 Jacek Pospychala CLA 2007-06-28 09:37:11 EDT
Created attachment 72682 [details]
tests

test case. hope you like it...
there is still issue with consistent saving, because newline characters are taken from "line.separator" system property, which may change on different systems.
Comment 16 John Arthorne CLA 2007-06-29 11:22:04 EDT
Assigning to me for review.
Comment 17 John Arthorne CLA 2007-07-09 16:33:09 EDT
Patch applied. Thanks Jacek!
Comment 18 John Arthorne CLA 2007-07-11 10:29:32 EDT
I should have caught this, but the test fails on linux and mac because it uses a Windows-specific path. I will fix up the test.
Comment 19 Krzysztof Michalski CLA 2007-08-10 07:25:35 EDT
Verified.
Comment 20 Szymon Brandys CLA 2008-03-11 09:50:52 EDT
*** Bug 217232 has been marked as a duplicate of this bug. ***
Comment 21 Szymon Brandys CLA 2008-04-30 04:46:31 EDT
Verified by code inspection.
Comment 22 Martin Oberhuber CLA 2010-08-17 05:26:44 EDT
CQ:WIND00152288

Most problems with read-only .project files and linked resources have been fixed by this, but bug 322821 has another special case (on Windows, due to case-insensitive file systems). I would appreciate feedback from others on the patch I attached there.