Bug 355049 - [Import/Export] Importing a project containing a linked resource will dereference that linked resource
Summary: [Import/Export] Importing a project containing a linked resource will derefer...
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: IDE (show other bugs)
Version: 3.6   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: 4.6 M5   Edit
Assignee: Andrey Loskutov CLA
QA Contact:
URL:
Whiteboard:
Keywords: bugday, helpwanted
Depends on:
Blocks:
 
Reported: 2011-08-18 05:42 EDT by Helmut J. Haigermoser CLA
Modified: 2016-01-12 14:47 EST (History)
9 users (show)

See Also:


Attachments
Exported Sample Project (45.99 KB, application/octet-stream)
2012-09-06 09:32 EDT, Martin Oberhuber CLA
no flags Details
patch based on John's suggest in comment 2. (780 bytes, patch)
2012-11-22 16:00 EST, Troy Bishop CLA
no flags Details | Diff
Proposed dialog changes (78.93 KB, image/png)
2015-12-04 08:46 EST, Andrey Loskutov CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Helmut J. Haigermoser CLA 2011-08-18 05:42:13 EDT
Build Identifier: I20110613-1736

In Eclipse 3.5 importing a project containing the section
	<linkedResources>
		<link>
			<name>Source</name>
			<type>2</type>
			<location>C:/Source</location>
		</link>
	</linkedResources>
into the workspace will result in the workspace project having the linked resource section as well.

However, starting with 3.6 this is broken, the linked resource is not remembered, and the linked resource folder gets copied into the project.

Reproducible: Always

Steps to Reproduce:
1. Import a project containing linked resources into your workspace
2. Choose to make a copy, or import from an archive
3. Check that the linked files, they will be hard copies now
Comment 1 Helmut J. Haigermoser CLA 2011-08-18 05:43:53 EDT
CQ:WIND00292723

*changing the Version to 3.6*
Please let me know what you think about this one, a technical comment from the platform team should clarify what is happening here, and if we can get it fixed or improved in 3.7.x/3.8

Helmut
Comment 2 Helmut J. Haigermoser CLA 2011-09-12 04:50:18 EDT
(In reply to comment #1)
> CQ:WIND00292723
> *changing the Version to 3.6*
> Please let me know what you think about this one, a technical comment from the
> platform team should clarify what is happening here, and if we can get it fixed
> or improved in 3.7.x/3.8
> Helmut

Hi @ll :)
Can we get a technical comment from the platform team?
Thanks Guys! :)
Helmut
Comment 3 Helmut J. Haigermoser CLA 2011-11-02 11:03:18 EDT
(In reply to comment #2)
> (In reply to comment #1)
> > CQ:WIND00292723
> > *changing the Version to 3.6*
> > Please let me know what you think about this one, a technical comment from the
> > platform team should clarify what is happening here, and if we can get it fixed
> > or improved in 3.7.x/3.8
> > Helmut
> Hi @ll :)
> Can we get a technical comment from the platform team?
> Thanks Guys! :)
> Helmut

*ping*
Can we get a technical comment classifying this issue?
Let me know if you need anything! :)
Helmut
Comment 4 John Arthorne CLA 2011-11-02 11:41:39 EDT
I can't reproduce this problem, using I20111027-1410.

I used "Import existing project" import wizard. I tried both with and without checking the "copy projects into workspace" option checked. In both cases the linked resource was preserved rather than copied. Can you try again with a newer build, or if you are importing differently please describe how you are doing it.
Comment 5 Szymon Brandys CLA 2011-11-02 13:50:55 EDT
I looked at Eclipse SDK 3.6 and 3.7 and I can't reproduce the issue either.
Comment 6 Helmut J. Haigermoser CLA 2011-11-03 12:42:51 EDT
(In reply to comment #5)
> I looked at Eclipse SDK 3.6 and 3.7 and I can't reproduce the issue either.

Thanks for looking into this guys! :)
You're of course right, I do some specific import/exporting, here is what I do:

1.) Create project with linked resource
2.) Export to archive file
3.) Import existing projects from archive file

Following these steps the imported project will have lost the linked resource information. Looking at the archive the .project file is still correct in there, the files from the linked location will have been added to the archive though...

Helmut
Comment 7 John Arthorne CLA 2011-11-03 13:58:29 EDT
Moving to UI where the Import/Export code lives. There are a few possible behaviours here:

1) Export traverses linked resources, copying the contents of all links into the archive. On import we create the external locations on disk that the linked resources point to, and import the various parts of the project into the different locations.
2) Export doesn't traverse linked resources On import we recreate the linked resource and point it at the existing directory on disk.
3) Export traverses linked resources, and imports them into the project as regular resources.

Currently we do 3). I'm not sure what behaviour you are expecting, but I would argue for doing 2). This would be consistent with how we treat linked resources in a source control system. The linked resource definition is exported/imported with the project contents, but the destination of linked resources is not touched.
Comment 8 Helmut J. Haigermoser CLA 2011-11-03 14:06:56 EDT
(In reply to comment #7)
> Moving to UI where the Import/Export code lives. There are a few possible
> behaviours here:
> 
> 1) Export traverses linked resources, copying the contents of all links into
> the archive. On import we create the external locations on disk that the linked
> resources point to, and import the various parts of the project into the
> different locations.
> 2) Export doesn't traverse linked resources On import we recreate the linked
> resource and point it at the existing directory on disk.
> 3) Export traverses linked resources, and imports them into the project as
> regular resources.
> 
> Currently we do 3). I'm not sure what behaviour you are expecting, but I would
> argue for doing 2). This would be consistent with how we treat linked resources
> in a source control system. The linked resource definition is exported/imported
> with the project contents, but the destination of linked resources is not
> touched.

Thanks John, sounds like option number 2 is what we should be doing...
Note that this used to work with Eclipse 3.5 so maybe option 3 was not always how we used to treat this?
Helmut
Comment 9 Martin Oberhuber CLA 2012-01-23 11:47:07 EST
Ping...

This issue has been reported by a customer of ours, and seems to be a regression ... any update ?

Thanks,
Martin
Comment 10 John Arthorne CLA 2012-01-24 10:27:38 EST
Available UI committers are scarce at the moment. I don't think anyone has looked at it. It sounds like the problem occurs on export. See ArchiveFileExportOperation#exportResource.
Comment 11 Martin Oberhuber CLA 2012-09-06 09:32:57 EDT
Created attachment 220784 [details]
Exported Sample Project

The problem occurs on import. 

Consider attached sample project. It includes a linked resource pointing 
   src/ -> T:\testsrc , 
and was exported with Eclipse 3.7.2.

Importing this project into Eclipse 3.5.1 does restore the linked resource at T:\testsrc . But importing into a later Eclipse restores the files inside the workspace / project under src/ and remove the linked resource information from the .project file.

The change of behavior is somewhat understandable in the case that the linked resource might point to a drive that does not exist, or is not writable. The problem with the modified behavior is, though, that
  (a) the exported project is not faithfully restored, and
  (b) there is no workaround restoring the archive faithfully, since no 
      external archivers would understand Eclipse linked resources, which 
      is needed here.

It looks like in order to restore previous behavior, the import would _first_ need to restore linked resource information, and _then_ traverse the resulting project structure.
Comment 12 Raquel Maldonado CLA 2012-11-22 10:35:54 EST
Is it possible to target this to 4.2.2, or... is there a bug similar to this one for that version? This is reproducible on Juno and is affecting our adopter product.
Comment 13 John Arthorne CLA 2012-11-22 10:56:54 EST
(In reply to comment #12)
> Is it possible to target this to 4.2.2, or... is there a bug similar to this
> one for that version? This is reproducible on Juno and is affecting our
> adopter product.

Would you like to have a try at creating a patch? I pointed at the relevant source code in comment #10. Here is a pointer in the git repository:

http://git.eclipse.org/c/platform/eclipse.platform.ui.git/tree/bundles/org.eclipse.ui.ide/src/org/eclipse/ui/internal/wizards/datatransfer/ArchiveFileExportOperation.java
Comment 14 Troy Bishop CLA 2012-11-22 16:00:53 EST
Created attachment 223889 [details]
patch based on John's suggest in comment 2.

For me, when I export a project archive from an environment with this patch applied and then re-import it as a project then the linked resource is successfully recreated.  I don't understand how or why it's successfully recreated (didn't investigate much), but it worked for me ;)
Comment 15 Troy Bishop CLA 2012-11-22 16:01:45 EST
(In reply to comment #14)
> Created attachment 223889 [details]
> patch based on John's suggest in comment 2.
> 
> For me, when I export a project archive from an environment with this patch
> applied and then re-import it as a project then the linked resource is
> successfully recreated.  I don't understand how or why it's successfully
> recreated (didn't investigate much), but it worked for me ;)

sorry, that should have said:

patch based on John's suggest in comment 7.
Comment 16 Raquel Maldonado CLA 2013-01-07 10:56:19 EST
(In reply to comment #13)
> (In reply to comment #12)
> > Is it possible to target this to 4.2.2, or... is there a bug similar to this
> > one for that version? This is reproducible on Juno and is affecting our
> > adopter product.
> 
> Would you like to have a try at creating a patch? I pointed at the relevant
> source code in comment #10. Here is a pointer in the git repository:
> 
> http://git.eclipse.org/c/platform/eclipse.platform.ui.git/tree/bundles/org.
> eclipse.ui.ide/src/org/eclipse/ui/internal/wizards/datatransfer/
> ArchiveFileExportOperation.java

Has the patch provided by Troy on comment #14 and comment #15 been tested?
Comment 17 Paul Webster CLA 2013-01-25 14:37:25 EST
(In reply to comment #14)
> Created attachment 223889 [details]
> patch based on John's suggest in comment 2.


Released as http://git.eclipse.org/c/platform/eclipse.platform.ui.git/commit/?id=fd05dadbcd7ee4e126cd2e44e1582c2a263afd14

Thanks Troy.

PW
Comment 18 Martin Oberhuber CLA 2013-01-29 05:38:44 EST
I'm afraid that this fix is not the right one.

Consider a project that has a linked resource folder pointing "to the outside".  Easy to reproduce by creating New > Folder > Advanced : Link to folder in file system, and choose a folder with some tree of files/folders underneath it.

Now I File > Export > Archive File. The export wizard shows a UI in which I can manually select files and folders to export. The tree in that UI includes linked resource folders as well as the files and folders beneath those.

When I'm offered to select files and folders under a linked resource, I expect the export wizard to actually export the data underneath those linked resources into my archive. This was the functionality in Eclipse up until today. No longer exporting anything below the linked resource (although the UI offers me to do so) is a regression IMO.

As I have said in comment 11, I think the problem (which was introduced as a regression between Eclipse 3.5.1 and Eclipse 3.6) is on import. The exported archive, which I have also attached as a sample, must include all the data that the UI wizard has offered to export (no loss of data - that's my acceptance criterion). 

The question is how to properly restore it on import, and there are multiple possible options doing so. From a user's point of view, IMO the ideal solution would be:
  1. Import the .project file first (restoring all Linked Resources)
  2. In case any remaining resources on import write into a Linked Resource,
     which points outside the project tree this may be unexpected and thus
     the user should be asked with a dialog:
     "This project includes Linked Resources which point outside the
      project tree. Do you want to write to those external locations [y/n]"

That solution would allow to faithfully import a project and restore it exactly as it was exported; or, only import those portions that remain underneath the project root and keep linked resources as-is.
Comment 19 Paul Webster CLA 2013-01-29 09:37:15 EST
I've reverted the fix for M5.

PW
Comment 20 John Arthorne CLA 2013-10-03 15:22:48 EDT
As I said in comment #7 there are multiple valid answers here. I still think Troy's approach of not exporting link contents is reasonable. It is an easy solution and it is consistent - there is no loss of data if you export the project, delete it, and then import it again. We don't traverse link contents on copy or delete so making export consistent with this is quite reasonable in my opinion (and matching for example how symbolic links work). 

Martin has a point that Troy's fix isn't entirely right. The UI lets the user select resources under the link which gives the impression that they can be exported. Maybe we should instead filter out the children of links when rendering the tree so they can never even be selected.
Comment 21 Eclipse Genie CLA 2015-12-02 10:57:26 EST
New Gerrit change created: https://git.eclipse.org/r/61796
Comment 22 Eclipse Genie CLA 2015-12-03 10:13:08 EST
New Gerrit change created: https://git.eclipse.org/r/61879
Comment 23 Eclipse Genie CLA 2015-12-03 10:13:14 EST
New Gerrit change created: https://git.eclipse.org/r/61878
Comment 24 Andrey Loskutov CLA 2015-12-03 11:50:10 EST
(In reply to John Arthorne from comment #20)
> As I said in comment #7 there are multiple valid answers here. I still think
> Troy's approach of not exporting link contents is reasonable. It is an easy
> solution and it is consistent - there is no loss of data if you export the
> project, delete it, and then import it again. We don't traverse link
> contents on copy or delete so making export consistent with this is quite
> reasonable in my opinion (and matching for example how symbolic links work). 

https://git.eclipse.org/r/61796

> Martin has a point that Troy's fix isn't entirely right. The UI lets the
> user select resources under the link which gives the impression that they
> can be exported. Maybe we should instead filter out the children of links
> when rendering the tree so they can never even be selected.

https://git.eclipse.org/r/61878
https://git.eclipse.org/r/61879

I've followed the John's proposal with the patches above.

Per default we should not offer linked resources to be exported - this makes Export/Import not symmetrical and in worst case (unfortunately our case) breaks the software because the projects imported back aren't properly linked.

I've added a new checkbox "Resolve and export linked resources" to the File->Export->General->File System and File->Export->General->Archive File wizards. 

This checkbox is *not* selected by default, and in this case the file tree also hides all linked resources from user.

Trying to export the project with linked resources in this case per default exports no linked resources (surprise) but if the .project file is exported too, importing such project back results in the exact 1:1 copy restored, with all the exported/imported links properly configured as before.

If the checkbox is selected, we see linked resources in the tree, can select and export them in a "resolved" state as before, resulting in a broken project with links replaced to the physical files after import (as before).

One of the arguments why we should not solve this on import (as proposed in comment 18) is that packaging every resolved link target into the archive could create (worst case) archives which could carry almost no "interesting" data but resolved link targets.

While I think that the implemented proposal is acceptable and good one I would like to know your opinion about the naming and initial state of the checkbox.

Naming: 
1) "Resolve and export linked resources" (current proposal)?
2) "Resolve linked resources"?
3) "Export linked resources"?

State:
1) "[]" - initially disabled (current proposal)?
2) "[x] - initialy enabled?

So what do you think?
Comment 25 Andrey Loskutov CLA 2015-12-04 08:46:17 EST
Created attachment 258460 [details]
Proposed dialog changes

See attached screen shot how the new proposal looks like
Comment 26 Andrey Loskutov CLA 2016-01-07 10:36:06 EST
(In reply to Andrey Loskutov from comment #25)
> Created attachment 258460 [details]
> Proposed dialog changes
> 
> See attached screen shot how the new proposal looks like

Ping. If there is no comments on the proposal, I'm going to merge it.
Comment 27 Sergey Prigogin CLA 2016-01-11 21:41:35 EST
(In reply to Andrey Loskutov from comment #24)
> Naming: 
> 1) "Resolve and export linked resources" (current proposal)?
> 2) "Resolve linked resources"?
> 3) "Export linked resources"?
> 
> State:
> 1) "[]" - initially disabled (current proposal)?
> 2) "[x] - initialy enabled?

I like the current proposal.
Comment 31 Andrey Loskutov CLA 2016-01-12 14:47:23 EST
@Sergey, Mickael: thanks dor review and feedabck. Merged.