Bug 6664 - [resources] Sym-link-like feature
Summary: [resources] Sym-link-like feature
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 2.0   Edit
Hardware: All All
: P2 enhancement (vote)
Target Milestone: 2.1 M5   Edit
Assignee: Platform-Resources-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 6883 8450 9508 9509 15335 15393 19111 22924 23243 24252 30691 66629 (view as bug list)
Depends on:
Blocks:
 
Reported: 2001-12-06 18:44 EST by Andrew McCullough CLA
Modified: 2011-12-29 15:04 EST (History)
36 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew McCullough CLA 2001-12-06 18:44:39 EST
I would like to see some generic "sym link" or "shortcut" concept in Eclipse.  
This would allow you to create different "views" into your code/resources/files 
without requiring you to maintain separate copies of the files.  If you change 
it in one place, that change is reflected everywhere. This would also allow you 
to create "mini-views" of your project, where smaller-granularity groups of 
classes/packages can be organized into projects, allowing the developer to 
organize his thoughts/work/etc at a sub-project granularity.

I think this could be a (relatively) simple concept with major functionality 
improvements. 


Here is how this came up:

NOTE:  This becomes much more of an issue is WSAD and not so much is "base" 
Eclipse


The various J2EE standards require code to be organized in different ways. For 
example, a WAR, EAR, EJB Jar, all have a particular structure.  WSAD allows you 
to "declare" what kind of J2EE app you are developing, and organizes the 
project accordingly. 

However, sometimes code is already organized another way, and it might be hard 
to change.  Example:  You have a project that has some vanilla Java code and 10 
EJBs.  You used to prefer Notepad/Emacs/JoeSmithEditor, and your "project" is 
organized in a very simple way:

Project
   srcfolder
   binfolder
   resources

This project is under source control, and changing the structure is not 
feasible.

You get a copy of WSAD for your birthday, and you decide to use it.  You 
realize it has all this nice integration for 
deploying/editing/creating/administering your EJBs and running them in a test 
App server.  Unfortunately, you can only use these features if you have your 
code organized in the standard J2EE format.  Even worse, this means creating an 
EJB jar for all 10 EJBs.  To do that, you have to create 10 EJB projects in 
WSAD.  But your code is already somewhere else -- in the large, simple, Java 
project.  If there was a link concept, you could create the EJB project(s) and 
just reference the code that was in the larger, simpler project.  


Does this make sense?

-Andrew
Comment 1 Marco Qualizza CLA 2002-01-09 10:50:59 EST
I whole-heartedly agree with this concept.  Because of the very situation
outlined in this bug report I've had to recommend that we hold off for a while
before adopting WSAD here. :-)  (Although we're closer to 40 EJBs... ;-) )

This would also help with the fact that the whole workspace concept is too
restrictive -- if I want to build to directory x, then x has to be part of my
project.  Eclipse already demonstrates that it truely doesn't need to depend on
having all files in one location -- Projects can be kept in folders external to
the Workspace -- all it needs is the knowledge of what the locations are.  Add
Symlinks, and Eclipse's flexibilty and usability would increase 10-fold. (And
add a caveat about using directories which are used outside of Eclipse...)

   - Marco Qualizza
Comment 2 John Arthorne CLA 2002-02-06 14:17:22 EST
*** Bug 8450 has been marked as a duplicate of this bug. ***
Comment 3 John Arthorne CLA 2002-02-06 14:19:05 EST
*** Bug 6883 has been marked as a duplicate of this bug. ***
Comment 4 John Arthorne CLA 2002-02-13 14:02:51 EST
*** Bug 9508 has been marked as a duplicate of this bug. ***
Comment 5 Carolyn MacLeod CLA 2002-03-11 16:44:37 EST
- See also the whole (long & useful) newsgroup thread entitled "Source folders".
- See also the newsgroup thread "How do I specify an external directory for 
classes?"
- See also another request in the newsgroup for this with title "build classes 
in seperate directory".
- Also, consider the fact that the people who wrote the C plugin did backflips 
to work around this problem in their plugin.
- Also, see these related PRs asking for external folders on the classpath:
  http://dev.eclipse.org/bugs/show_bug.cgi?id=8655
  http://dev.eclipse.org/bugs/show_bug.cgi?id=3074
This whole concept of "Eclipse is too possessive about source files" is 
actually quite important for a large number of people. We need to think about 
how to manage/use source files from any random location on the build path. All 
of the other IDE products do this. I think the priority of this PR should be 
higher.
Comment 6 Cedric Beust CLA 2002-03-25 19:28:20 EST
I have a requirement that would be solved by this feature, although my need 
could also be addressed with the simple possibility to be able to selectively 
exclude some source folders from Eclipse's compilation.

In short, I want to be able to express the following:

Source folder:  src/
Include:  com.beust.a
Exclude:  com.beust.b, com.beust.c

This is similar to what ant does, and would therefore allow a more seamless 
migration of projects currently built by ant to Eclipse.
Comment 7 Martin Möbius CLA 2002-04-09 09:34:20 EDT
A workaround for Win2k are junctions
from the news:
>Don't despair. You have true symlinks on winnt/2k, checkout
>http://www.sysinternals.com/ntw2k/source/misc.shtml#junction.
>Barry Kaplan - "Re: Excluding some source directories"):
Comment 8 Carolyn MacLeod CLA 2002-04-09 14:17:45 EDT
See also the newsgroup posts:
- Rigid project directory structure?
- CVS projects
Comment 9 Carolyn MacLeod CLA 2002-04-10 18:13:14 EDT
Something else to keep in mind when considering the implementation of this 
feature: (from the newsgroup)

From: "eliuth" <pomar@us.ibm.com>
Subject: Remote projects for JDT, PDE
Date: Wednesday, April 10, 2002 4:59 PM

Hi. This could be a very naive question...

The CDT plugin provides a server part to manage remote projects. Looks
like you can compile code on a romote host machine and run it from an
eclipse workstatation client version.

Is there a way that the same type of scenario be done with JDT and PDE
plugins? The FAQs say you can point your project to a specific JRE.
However, the wizard that allowes this doesn't provide for a host name or
port connection to a remote host.

Thanks for any info or insights,
Eliuth
Comment 10 John Arthorne CLA 2002-05-15 19:00:54 EDT
*** Bug 15335 has been marked as a duplicate of this bug. ***
Comment 11 DJ Houghton CLA 2002-05-23 15:07:29 EDT
Would require too major of a re-work to do this late in the cycle.
Investigate post 2.0.
Comment 12 DJ Houghton CLA 2002-06-04 14:27:35 EDT
*** Bug 19111 has been marked as a duplicate of this bug. ***
Comment 13 David J. Orme CLA 2002-08-27 10:46:11 EDT
Bob Foster had some good thoughts in the newsgroup thread titled "Call for
screenshots":

--

[when complaining that you can't double-click a Java file in Explorer and have
it open in Eclipse...]

Everybody has their favorite editor. Mine right now just happens to be
Eclipse. ;-}

> I think the real problem is, how should eclipse handle those foreign files?
> It can certainly not compile them since nothing is known about source and
> library paths etc. How would the user know (you and I would certainly know
> which file is which, but how do I tell my Junior programmer? :)

You've hit the nail right on the head! That is the real problem, or rather
both of them. Don't know all the answers, but to "think out loud" for a
moment...

First, editors and other views need to make a distinction between external
and internal resources. There isn't any reason why TextEditor can't offer
all its capabilities to external resources, but there is a reason why
CompilationUnitEditor can offer only a subset. There may be a reason why the
Packages view can't show external resources; there may not be a reason why
the Navigator view can't. That should be the choice of the individual
editor/view, not the platform.

Second, there should be a clearly visible indication which resources are
internal and which external. This will help users create a model that
"explains" why they behave differently.
Comment 14 DJ Houghton CLA 2002-08-27 11:06:06 EDT
*** Bug 9509 has been marked as a duplicate of this bug. ***
Comment 15 DJ Houghton CLA 2002-08-27 11:07:06 EDT
The above issues are also discussed in bug 6637.
Comment 16 Jonathan Gossage CLA 2002-08-27 20:12:04 EDT
The file placement issue became a showstopper regarding the introduction of
Eclipse in our organization. There was far too high an investment in tolls
built around Ant and the current structure for Eclipse to be adopted at
present.

Personally, I believe that it should be possible to root a project in any
existing directory and then define a view within Eclipse that maps a portion
of the physical directory tree as the logical structure for an Eclipse
project. Such a capability would greatly simplify the process of importing
existing environments into Eclipse and would allow the preservation of
useful existing tools and procedures.

At this point in Eclipse's development I feel that broadening the user base 
should have a high priority and removing this obstacle would help to encourage 
a higer adoption rate.

A very preliminary investigation suggests that it might be possible to 
implement this functionality as a plugin, but I need to do a lot more digging 
before I would commmit any resources to the idea.
Comment 17 Cagatay Kavukcuoglu CLA 2002-08-28 01:41:58 EDT
From newsgroup posting:

Bob Foster wrote:
(snip)
You've hit the nail right on the head! That is the real problem, or rather
both of them. Don't know all the answers, but to "think out loud" for a
moment...

First, editors and other views need to make a distinction between external
and internal resources. There isn't any reason why TextEditor can't offer
all its capabilities to external resources, but there is a reason why
CompilationUnitEditor can offer only a subset. There may be a reason why the
Packages view can't show external resources; there may not be a reason why
the Navigator view can't. That should be the choice of the individual
editor/view, not the platform.

Second, there should be a clearly visible indication which resources are
internal and which external. This will help users create a model that
"explains" why they behave differently.

Bob


This troubles me. I think platform has a definite say in what is considered a
resource. Using VAJ wised me up to the evils of making some parts of the project
internal and some other parts external. I think it would be a step back to
introduce internal/external duality for resources (This is not the same thing as
local/remote resource issue though - which I think should be possible to have).

I like how Eclipse resource workspace concept works. It's very clear what's
included in a project. I'm happy with the lack of sym-links in workspace though
additional functionality to map resource folders in workspace to arbitrary
folders in file system is needed (this can be done without breaking the existing
semantics of the workspace).

On the other hand, I agree that it is desirable for existing editors to provide
all or a subset of their functionality for arbitrary files. I think Platform
Resource can provide rudimentary support for unmanaged storage in arbitrary
paths (with a IWritableStorage interface and a simple implementing class
perhaps) and Platform UI can provide a file explorer view similar to the one in
SWT examples. With these available, any editor that understands how to save its
content to a writable storage editor input could be used on arbitrary files.
Comment 18 Bob Foster CLA 2002-08-29 00:33:51 EDT
What you say doesn't trouble me. ;-} In fact, except for terminology (I don't
come from VA background and may accidentally kick up the bones of old
controversies) we may be saying the same thing. I agree that the items you list
are needed.

- Ability to open "remote" resource in an editor.
- Ability to "map resource folders in workspace to arbitrary folders in file system"

That's not an exhaustive list, but it addresses the top two issues.

Don't have a problem with proposed "IWritableStorage" implementation, either.
That's basically what I am about to do for my plugin. A platform solution would
be so much better.
Comment 19 DJ Houghton CLA 2002-09-10 12:11:43 EDT
Reopening for investigation.
Comment 20 Kent Johnson CLA 2002-09-13 13:22:15 EDT
*** Bug 23243 has been marked as a duplicate of this bug. ***
Comment 21 Daniel S.V. MacDonald CLA 2002-09-13 14:32:27 EDT
How about expanding the concept of the "Working Set" to select folders and 
files for the build process?
Comment 22 Robert Klemme CLA 2002-09-19 05:00:54 EDT
just some thougts on this topic:

first of all you can work good with several projects.  it's only that in this 
case you better don't work with the "super project" as you call it.  normally 
if these are real sub projects then i would expect them to compile 
independently and have an acyclic dependency graph connecting them.  we work 
that way, that all sub projects are independently build into a jar which is 
then integrated into the other projects.   this does make sense if the sub 
projects are quite stable.  if they are not (yet) you could incorporate the 
dependencies into eclipse's projects.

for a release build i would use a clean location anyway where i extract the 
complete super project from cvs to make sure that there are no files forgotton 
to check in etc.

the introduction of those links can reduce redundancy on the file level but it 
can create more confusion as well.
Comment 23 Macon Pegram CLA 2002-09-19 20:58:24 EDT
Use case info (originally posted in eclipse.tools newsgroup) per request of John
Arthorne

I'm a huge fan of Eclipse, and have been using it for awhile.  I've
recently joined a project at work which has been around for quite awhile. 
They have expressed an interest in using Eclipse as their IDE, but we're
running into one glitch.

Source code is currently arranged in the project as follows:

foo/source             - Base directory for source
foo/source/com         - The beginning a bunch of packages (com.xxx.xxx.)
foo/source/batch       - More code which is NOT in a "batch.xxx"
package structure

Now I'll be the first to admit this, isn't the best way to organize
one's code, but it's what I have to deal with and I don't have the
option to change this structure at this time.  We're doing our
development under WIN NT 4 (so the previous suggestions in the message
board of using the W2K tool Junction to create links will not work for
us).  

When I setup the project to use "foo/source/batch" as the base
directory for source, I can deal with the code in that directory just
fine, but this prevents me from pointing the project to "foo/source" as
an additional source directory since "batch" is nested within
"foo/source", so I can't work on the code in the "foo/source/com"
directory.

If I try to do the reverse, where I tell Eclipse that "foo/source" is
my source directory, Eclipse has a fit on compilation because it wants to
compile all the java code in the "foo/source/batch" directory as if it
was in a "batch.xxx" package.  

Are there any work arounds anyone can suggest?  This is the only real
reason we can't use Eclipse on this project.  Aside from this one issue
Eclipse would be a MUCH better tool than any of the others currently being
used on the project. 

As I see it there's two possible solutions (but I don't know how to
implement either one):
- 1. Define "foo/source/batch" and "foo/source" as source
directories (with the  corresponding package mapping relative to these
directories).    This seems pretty unlikely given the way Eclipse does
things.

- 2. Is there some way to get Eclipse to ignore compilation in the
"foo/source/batch" directory when I define "foo/source" as my
project's source directory?  I know you can FILTER seeing
"foo/source/batch", but I want to be able to actually have Eclipse
FILTER compilation of this directory.  That would allow me to set up a
separate Eclipse project to deal with the contents of the
"foo/source/batch" tree.  (Which would be a perfectly acceptable
solution for my needs).

------- 
This wasn't in the original newsgroup posting, but after reading the comments
below, I want to pile on as far as the need to be able to compile OUTSIDE of the
Eclipse project folder.  Here's the value of doing this to me and my team:
- We use Clearcase on which me connect to via NFS over NT.  
- The Clearcase VOB can be VERY slow at times. 
- Since the source is in the VOB, and Eclipse insists on compiling into my
project folder, I have to wait a considerably long time for both VOB based reads
and writes.  If I could cut this down to just READS I'm sure we'd greatly speed
up compilation time.   (As is evident when we use ANT scripts to compile to
targets outside the VOB).

These two issues (compilation in the project folder and dealing with the
location of source folders) are the only two complaints I have with Eclipse.
Comment 24 John Arthorne CLA 2002-09-23 16:33:53 EDT
This bug report and its duplicates have splintered into discussions on a variety
of issues.  In an attempt to pin down a particular problem that needs to be
solved, we have written a document to describe what we are calling the
"inflexible project structure" problem.  Please see my recent post in the
eclipse.tools newsgroup (09-23-2002, 3:45 PM EST) for more information, and
provide feedback in that forum. The post is here:

news://www.eclipse.org:119/3D8F6F3D.5020609@o_ti.com

The problem definition document is found here:

http://www.eclipse.org/eclipse/development/inflexible-projects-problem.html

I'd like to avoid having this bug report also include the problem of opening
editors on files that are not in the workspace for quick browsing/editing.  This
is really a separate issue that is discussed in other bug reports.  For example,
see bug 22478.
Comment 25 Bob Foster CLA 2002-09-23 17:31:22 EDT
"I'd like to avoid having this bug report also include the problem of opening
editors on files that are not in the workspace for quick browsing/editing.  This
is really a separate issue that is discussed in other bug reports.  For example,
see bug 22478."

How about designating one bug to track this problem?

Comment 26 John Arthorne CLA 2002-09-23 18:05:48 EDT
> How about designating one bug to track this problem?

I would like to, but this issue falls outside my "bugzilla jurisdiction".  This
is a known issue without a definitive home in bugzilla (that I know of).  I'll
propose the following:

bug 4922: for associating eclipse against files with the O/S (so double-clicking
the file in your O/S will open the file in eclipse)

bug 21973: for using an eclipse editor to edit files that are not in the
workspace (eg, using File->Open).

A fix for bug 21973 would be required before addressing bug 4922.  The
distinction is that this bug (6664) is about adding files into the workspace,
and those bugs are about editing files that are not in the workspace (ie, the
user doesn't want them to live in any project, they just want to view them from
within the eclipse UI). I hope that helps reduce the bug report swirl ;)

Comment 27 Jonathan Gossage CLA 2002-09-24 18:11:48 EDT
It seems to me that the underlying goal that needs to be achieved is to 
decouple the logical representation of a project within the workspace from the 
physical realization of the project in 

the persistent backing store for the project.

There are three areas I can see where there are significant problems:

	Location and name of the project file
	Definition of the directory subtrees that contain the sources for an 
individual project.
	Definition of the directory subtrees that contain the project output 
files.

The current practice of using the name ".project" as the name of the project 
file carries an unfortunate side effect when trying to adapt existing project 
structures to meet Eclipse 

requirements. It forces the project metadata to be stored in a directory that 
may exist solely to provide a root for the project. Obviously this is not a 
problem for new projects but it can 

be problamatic for projects that have been maintained using a different 
toolchain.

It should be possible to allow each set of project metadata to be uniquely 
named and to reside at an arbitrary location. To make this possible, it will 
be necessary to maintain an explicit 

registry of some kind that can locate all the project files. One possiblity 
would be to extend the workspace metadata to store this registry.

The second and third problem areas might be handled in the following way. The 
project metadata should maintain two collections, one for sources and the 
other for project output. Each 

collection would contain one or more directory patterns which would define the 
portion of the physical persistent directory structure that forms part of the 
project tree.

A example of a directory pattern might be:

  /workspace/org/eclipse/some-project1/*/sub-directory1

where the * indicates that all directories under some-project should form part 
of the tree defined by this pattern. Everything except * should match 
according to the case rules for file 

names on the backing file system. Thus if the following physical directory 
structure existed

workspace
  org
    eclipse
      some-project1
        component1
        component2
           sub-directory1
        component3
           sub-directory2
      some-project2
        component1
        component2

the above pattern would result in the following directory tree
workspace
  org
    eclipse
      some-project1
        component1
        component2
           sub-directory1
        component3

Note that sub-directory2 under some-project1/component3 would not be included 
in the project, nor would some-project2 or anything under it.
In reality * should probably be a placeholder for the set of file selection 
patterns understood by the underlying operating system or for an equivalent 
GUI based selection mechanism.
Patterns would be applied when a file is loaded into the workspace from 
backing store and when new files and/or directories required to maintain 
package structure are created, or 

when classes are refactored into different packages.

This proposal would require a number of changes to the project wizards. First 
it would probably be prudent to define projects with these capabilities as new 
project types so as to not 

interfere with existing projects. It would be necessary to specify the 
location and name of the project file, and a tab would be required to allow 
the project structure to be created from 

the backing store.

This proposal is nowhere near complete and is offered simply as a possible 
starting point. 
Comment 28 DJ Houghton CLA 2002-09-27 13:35:54 EDT
*** Bug 15393 has been marked as a duplicate of this bug. ***
Comment 29 DJ Houghton CLA 2002-09-27 15:43:45 EDT
*** Bug 22924 has been marked as a duplicate of this bug. ***
Comment 30 DJ Houghton CLA 2002-09-27 15:43:59 EDT
From bug 22924:

I have a project called foo which is under the directory d:\home\foo.

Now this is a pretty big project with a lot of files. I use this project only
occassionally. However there is one folder under foo called "bar" which has 
some
files which I work a lot with. 

The problem is that if create a new project called bar, eclipse does not allow
me to create it since bar is already a directory of foo (even though I have the
project foo closed). 

I think this is very limiting because in my case the project foo is a very big
project and I'd rather not open it and consume ton of resources if I am working
in a subset of the project. In my view eclipse should allow creating
sub-projects even if the directory is a part of another project.

Comment 31 John Arthorne CLA 2002-10-01 10:23:45 EDT
*** Bug 24252 has been marked as a duplicate of this bug. ***
Comment 32 cdowns CLA 2002-10-06 01:17:36 EDT
Thank you submitters for so elequently stating this problem.  I add my voice 
to your chorus in hopes that an 'external build path' will soon become a 
feature in what is otherwise an outstanding IDE. 
Comment 33 Marcus Beyer CLA 2002-10-13 11:47:16 EDT
Another good argument for an 'external build path' is the possibility of using 
a Ramdrive. This is what I made to Ant and it saves neary 30 percent of compile 
time. I imagine "scrubbing the output folder" should then be done in a second.
Comment 34 John Arthorne CLA 2002-10-24 18:53:39 EDT
A proposed solution document has been posted here:

http://www.eclipse.org/eclipse/development/flexible-projects-proposal.html

Note that the scope of the problem addressed by this solution is described in
the document I mentioned in my 09/23 comment.  This solution does not address
ALL the issues raised in this bug report.  If you're interested in another
issue, you're CC'd on the wrong bug report.

Please provide feedback by following up to my recent posts on the eclipse-dev
mailing list or the eclipse.tools newsgroup, so that a wider audience can
participate (although this bug report has an impressively large audience already
;)).
Comment 35 Sonia Dimitrov CLA 2003-01-31 09:43:47 EST
*** Bug 30691 has been marked as a duplicate of this bug. ***
Comment 36 John Arthorne CLA 2003-02-04 11:55:57 EST
Fixed.  All solutions in the proposed solution document have been implemented. 
To give a brief summary:

- you can now link folders and files directly below projects to arbitrary
locations in the file system.
- linked resources can overlap each other, or other projects
- JDT source folders now have exclusion filters for ignoring sub-folders that
you don't want to compile
- JDT source folders now support nesting
- JDT source folders and build output folder can be linked resources
- You can now have one output folder for each source folder in a java project.
Comment 37 John Arthorne CLA 2004-06-16 12:43:53 EDT
*** Bug 66629 has been marked as a duplicate of this bug. ***