Bug 245412 - [resources] Support physical nesting of projects
Summary: [resources] Support physical nesting of projects
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 3.4   Edit
Hardware: All All
: P3 enhancement with 44 votes (vote)
Target Milestone: ---   Edit
Assignee: Platform-Resources-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: investigate
: 463355 (view as bug list)
Depends on: 44967
Blocks: 35973 246221 247647 251370 251408 253898 343584 398523 412689 441165
  Show dependency tree
 
Reported: 2008-08-27 12:42 EDT by Martin Oberhuber CLA
Modified: 2022-11-10 11:11 EST (History)
130 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Oberhuber CLA 2008-08-27 12:42:18 EDT
+++ This bug was initially created as a clone of Bug #35973 +++

I'm forking this specific request out of the way too broad discussion on bug 35973 in order to ask for comments and possibly work towards implementation.

At least in the C/C++/Compiled Language scope, we often have to deal with large source trees and legacy build systems, which cannot change easily. The choice of what parts of the tree need to go into a Project, however, is related to the type of contents and natures of the files. For instance, there could be a common root folder that's best mapped to a C/C++ project, but in a subfolder there could be some models that should go into a Modeling project, and yet another subfolder could hold some Java stuff.

Maven build structures seem another example where physical nesting of projects would be helpful.

It's important that Eclipse is able to adapt to customer's need and the existing file structure, rather than forcing the customer to make workarounds. Nested projects are a first and hopefully simple but powerful step on that path (which might end with more flexible resources as per bug 229633, or be complemented with non-root .project files as per bug 78438).

I'm suggesting the following, which is a simplification of my original suggestion on bug 35973 comment 103:

(1) The Workspace knows the list of project root locations, regardless of
    whether the project is open or not. It's sufficient that the project
    is added to the Workspace.

(2) Every traversal of a project tree (IContainer) checks whether a child
    container is actually a project root. If yes, traversal stops at this
    point and an icon similar to a "Linked Resource" which points to the
    actual project is displayed in spite of the child container.

I think that this mechanism is fully backward compatible with what we have today as long as no physically nested projects are added to the same workspace (which is not supported today). For the case of having the nested projects in the workspace, consistent behavior is exposed.

Note that my proposal still presents the physically nested projects in the
current (flat) Eclipse Project list. Also, it doesn't solve some other issues from bug 35973 (such as name clashes / namespaces, logical nesting or setting inheritance). I believe that these should be dealt with separately.

The suggestion is very powerful though, since it allows for easy adding of new project types/natures below an existing root; different version control systems inside one root; hiding subtrees by means of masking them with a project; team-sharing of subtrees with fine granularity; and more.
Comment 1 James Blackburn CLA 2008-09-17 10:11:36 EDT
(In reply to comment #0)

> At least in the C/C++/Compiled Language scope, we often have to deal with large
> source trees and legacy build systems, which cannot change easily. The choice
> of what parts of the tree need to go into a Project, however, is related to the
> type of contents and natures of the files. For instance, there could be a
> common root folder that's best mapped to a C/C++ project, but in a subfolder
> there could be some models that should go into a Modeling project, and yet
> another subfolder could hold some Java stuff.

Is this not covered by bug44967?  In 3.4 you can create multiple projects which have overlapping locations -- i.e. you can create an outer Project which may not have any exciting natures, and separate projects for your modelling, c, etc.  

Admittedly resources that exist in more than one place in the workspace aren't treated terribly well -- see bug247647, but Projects which are physically nested would appear to be supported.
Comment 2 Martin Oberhuber CLA 2008-09-17 12:53:36 EDT
(In reply to comment #1)
The problem with physically nested projects (as covered by bug 44967) is that their semantics are not clearly defined, some operations work while others do not. Clients are not prepared to receive notifications about resource deltas twice for the same item, or to have and editor for one and the same file in too different project contexts (foo and bar).

Here's another example where the UI doesn't support physically nested projects properly: With the New Project Wizard, you can create projects 
  (workspace)/foo
  (workspace)/foo/bar
Now, delete both projects from the workspace (keep files on disk). Then Import existing projects into filesystem. Only foo is shown, with a warning that "some projects were hidden".

Or take the team providers. Notifications, Decorators and markers will typically fail when you try to commit both foo and bar in one operation. Or, you get "Resource is out of sync with the file system".

I personally like the concept that in the normal case, project trees are *not* allowed to overlap and only linked resources can be used to create aliases. Perhaps the summary of this bug should be changed into

  "Physically nested projects should not introduce duplicate resources"
Comment 3 James Blackburn CLA 2008-09-17 13:12:12 EDT
(In reply to comment #2)
>   "Physically nested projects should not introduce duplicate resources"

:) I think we're on the same wavelength.  

The bugs I've filed elsewhere against nested projects / linked resources (bug247647, bug246221) are just specific examples of things that go wrong with the current model.
Comment 4 Martin Oberhuber CLA 2008-10-30 15:36:45 EDT
Bug 251370 and bug 251408 are also issues that just occur due to overlapping resources in nested projects. Masking out nested projects from their container (as this bug suggests) would therefore fix those two as well.

In case anybody is concerned about workspace backward compatibility when the resources of physically nested projects do not appear as duplicates in their container project any more, I believe that there is a simple migration path: Assume that project B is nested inside project A, then after a fix for this bug masks out project B, all you need to do is create a linked resource folder at the location, pointing into project B.

Nested resources will be available to the container A again, but in a more organized and (hopefully) better supported manner.
Comment 5 Martin Oberhuber CLA 2008-10-30 15:49:28 EDT
Marking as "depends on" bug 44967, because that one originally allowed physically nested projects but didn't clearly specify their semantics (leading to all the other issues).

This bug basically follows up on bug 44967, rationalizing the semantics by allowing physical project nesting without producing overlap (by masking out the nested project's folder).
Comment 6 James Blackburn CLA 2008-12-05 08:37:21 EST
How is this going to work from the team provider's POV?  Given that, in general, team providers don't support linked resources with these changes will users have to synchronize both project separately?  Currently a user (+/- a few patches to core.resources) can synchronize on the outer project and have resources in both projects update correctly.
Comment 7 Martin Oberhuber CLA 2008-12-05 16:31:09 EST
Yes,

my initial thought was that both projects would need to be synchronized separately. Selecting a group of projects for team synchronization together is not uncommon, and working sets help in this scenario. Remember that in the normal case, the projects (although physically nested) will be shown side by side.

I do believe, though, that on top of this initial behavior exposed by Core Resources, it should be possible to put advanced behavior on a higher layer. By means of project references, the projects could be displayed nested again -- the "knowledge" of their relationship is there. Or, on the UI layer, when you sync the outer project, a dialog could ask "do you want to sync nested projects too".

The important point here is that on the lowest level, the projects are independent. We still have the chance to run operations on a group of projects on higher levels.
Comment 8 Neale Upstone CLA 2009-07-13 06:39:01 EDT
Martin,

Thanks for grasping this, it's a problem for us end users that gets more complex as more tools and plugins come along.  It would certainly help to define the semantics of nesting, and I'd certainly agree that team providers give a good set of use cases.

My straw man use case would be "native" Maven project support:
- Can check out arbitrary subtrees from a large nested arrangement
- Subtrees can be checked out into workspace without need for any .project files ever being present.
- .project and pom.xml (and any other pluggable project definition abstraction, e.g. netbeans proj file, IDEA .iml) files within tree are detected and offered to be "opened".
- Navigator views such as Package Explorer support new view arrangements, adding "Nested paths" to the "Top level elements" sub-menu (in this case, each checked out subtree would show as the top).
- All detected project definitions are shown as either open or closed in the flat view (Top Level Elements -> Projects).

I'm currently working around this by having overlapping subtrees checked out.  One for the root of my maven project, and then each project within it.  Naturally I come up against the "Resource out of sync" issue.



Comment 9 Jörg Thönnes CLA 2012-11-06 04:18:45 EST
Martin, I tracked this bug from browsing nearly all comments of bug #35973.

But I am still confused:

We are starting to use Maven more and more (which is quite standard nowadays) and using the M2E integration. But the Package Explorer (and other views) still present the Maven projects in a flat view. But we would like to see a hierarchical view.

I do understand that you separated all concern according to different concepts. But since I am an end user, I do not understand and cannot care about these details.

Therefore my end user case is still "Display and manage the nested project structure of Maven project visually in the UI".

It is very difficult to see any progress here and our developers actively considering to move to another IDE (IDEA or NetBeans).

So where could I track the current state of affairs and what is *really* planned for the next E4 release?

Many thanks, Jörg
Comment 10 Mauro Molinari CLA 2012-11-06 04:44:47 EST
Just my 2 cents...

Apart from Maven, another build tool which is gaining popularity (thanks to its excellent features) is Gradle.

In my team we're using Gradle more and more and the problem of handling nested project is there, too. Gradle supports both a hierarchical layout for multiprojects and a flat layout, but the hierarchical layout has many advantages when working with huge multiprojects (20+ subprojects, for instance) if you consider the fact that the whole multiproject is usually shared on a SCM repository, so operations like switch, merge, update can then be easily done from the root project rather then on each single subproject.

Right now, we're handling the complexity of using Eclipse on nested projects structures using the Spring Tool Suite Gradle plugin, which handles the following:
- provides an "import" wizard that supports nested projects (the built in "Import | Existing projects into workspace" does not)
- provides the ability to automatically set up resource filters to filter out subprojects from the parent project view in the navigator views (Package/Project Explorer); projects are shown as siblings in Eclipse, but this is acceptable for us

Moreover, we are forced to checkout the multiproject with an external SCM client (using SVN) and then import it in Eclipse, because checking out a hierarchical project structure directly from Eclipse is not possible.
Lastly, I am discussing with Alexander Gurov of the Subversive project in order to see if something can be done to resolve some glitches we encounter when working with SVN on these nested projects, caused by the fact Eclipse does not support the concept of nested projects.

Of course, handling all of this complexity is not straightforward and I would appreciate very much if Eclipse one day will support such scenarios out-of-the-box.
Comment 11 Martin Oberhuber CLA 2012-11-06 08:54:21 EST
Reassigning to Inbox, as I'm not aware of any work going on right now.

Thanks for the comments regarding Gradle. The approach sounds promising to me - in fact it emulates with resource filters what I'm suggesting in comment #0.

Szymon, do you think some of this could move into Core/Resources ?
Comment 12 Mickael Istria CLA 2013-06-24 07:55:15 EDT
Here is a suggestion of approach that would make it easier to deal with modular projects in Eclipse, in term of UI
* All children of Workspace Root are projects.
* Resource Tree could show a project as a child of a Folder or a Project (generally, of a Container)
* Nested project that are closed would appear with the closed project (blue folder) icon, and would behave like Folder and list contents. So it's still easy to manipulate file system without having builds and other project mechanism running.
* Open nested project would appear like a project, but inside the tree. They would list the logical parts of the project (Class1path and so on).
* One could allow to "detach project view from parent" on nested project. In that case, the nested project would appear as a top-level project in Workspace, and the "parent" tree would show a leaf for this project that would actually be a link to the same project in Workspace Root (we could think about a "Go To Project" UI entry for easier navigation.

WDYT ?
Comment 13 Max Rydahl Andersen CLA 2013-06-25 06:17:43 EDT
I'm not following your suggestion - where and how would this be shown and what defines which nodes are shown if they are not projects ?
Comment 14 Mickael Istria CLA 2013-06-27 11:29:40 EDT
Here is a draft to support nested projects (using Path instead of names in workspace) in platform resources (core) https://git.eclipse.org/r/#/c/14108/
Comment 15 Mickael Istria CLA 2013-06-27 13:16:16 EDT
Here is a quick video of how things behave with the proposed change: https://vimeo.com/69255214
Comment 16 Neale Upstone CLA 2013-06-27 15:18:33 EDT
Nice work Mickael.  The video makes for inspiring viewing.

Do you have any thoughts on how to resolve the JDT issues you mention in your patches?
Comment 17 Mickael Istria CLA 2013-07-01 02:38:44 EDT
(In reply to comment #16)
> Do you have any thoughts on how to resolve the JDT issues you mention in
> your patches?

I didn't look deep into JDT, but I think the general issue is how to implement the Resource#getProject method now that there are nested projects. So far, it still reference the "root project" which is the direct child of the workspace root. I should give a try to change implementation in order to return the project itself for a nested project, and the parent project (which can be a nested one) for any other resource.
That's what this change is all about: trying to remove from the concept of "Project" the fact that it is assumed to be a direct child of workspace root.
Comment 18 Mickael Istria CLA 2013-07-15 02:49:00 EDT
Any chance my proposal could be reviewed by a committer on Platform/Resources so we have some time to polish it and hopefully make it part of Eclispe 4.4 ?
Comment 19 John Arthorne CLA 2013-07-17 16:03:03 EDT
(In reply to comment #18)
> Any chance my proposal could be reviewed by a committer on
> Platform/Resources so we have some time to polish it and hopefully make it
> part of Eclispe 4.4 ?

The proposal changes some clearly specified characteristics of the current resource API, for example that IProject is only found as a direct child of IWorkspaceRoot, and that if you ask for the children of a folder you will only receive files and folders as a result. As you noted this breaks assumptions in JDT and likely in many other plugins as well.

Your video is inspiring though, and I thought while watching it that we could come fairly close within the bounds of current API:

 - Add a "Convert to Project" command on folders in the UI. Maybe this only appears on folders that already contain a .project but that is debatable
 - When invoked, the command creates a new project rooted at that folder, and adds a resource filter to the existing project to avoid the duplication
 - An inverse command could appear on projects that are physically nested, to convert back to a folder. This would delete the project (not deleting contents), and remove the filter on the parent project so it would reappear as a regular folder.

As a further step, the "existing project" import wizard could be implemented to do this step automatically on all nested projects, or offer to create projects for folders with nested .project and add the necessary filters to remove the duplicate trees. Maybe it could even offer to automatically create a working set for all nested projects so you could get logical grouping of those projects in the Explorer/Navigator views.

While this isn't quite as slick as your proposal, it has the major benefit of not breaking existing plugins and provides similar capabilities to your prototype. My general feeling is that we allow physically nested projects today, but they are difficult to use and often the resulting duplicate resource trees are not desirable. I see a lot of room for usability improvements through use of resource filters, working sets, and additional commands for doing conversion back and forth without breaking changes for all existing clients.
Comment 20 John Arthorne CLA 2013-07-17 16:09:09 EDT
While playing around with current UI, I see the "existing project" import wizard doesn't even allow you to create/import a project that is physically nested in another project. You can still do it, but you have to use File > New Project instead of Import command, which is painful and doesn't support creating multiple projects at once. I think even changing this import wizard to allow importing nested projects, and add an option to automatically filter overlapping content from the parent project would be a big improvement.
Comment 21 Igor Fedorenko CLA 2013-07-17 16:18:47 EDT
I can't talk for other IResource API consumers, but for maven integration we don't need nested projects per se. What we need is ability to associate builders+configuration to possibly overlapping folders within projects. For example, a typical maven project looks something like this

project/
  pom.xml
  moduleA/
    pom.xml
    src/main/java/
    src/test/java/
  moduleB/
    pom.xml
    src/main/java/
    src/test/java/

So ideally, I'd like to represent this entire structure as a single workspace IProject rooted at project/ but have four JDT builder instances enabled for this project -- to compile sources in moduleA/src/main/java/, moduleA/src/test/java/, moduleB/src/main/java/ and moduleB/src/test/java/ source directories.

I think this can be introduced as an evolution of existing IResource API, i.e. new clients will work with existing projects, but old clients will not work with new projects, which I believe is totally acceptable. You can't make an omelet without breaking some eggs after all.
Comment 22 Mauro Molinari CLA 2013-07-18 03:25:30 EDT
(In reply to comment #19)
> The proposal changes some clearly specified characteristics of the current
> resource API, for example that IProject is only found as a direct child of
> [...]
> 
> Your video is inspiring though, and I thought while watching it that we
> could come fairly close within the bounds of current API:
> 
>  - Add a "Convert to Project" command on folders in the UI. Maybe this only
> appears on folders that already contain a .project but that is debatable
>  - When invoked, the command creates a new project rooted at that folder,
> and adds a resource filter to the existing project to avoid the duplication

I would like to add my 2 cents to the discussion.

I understand that supporting physically nested project would cause a breaking change, but I think that time is mature to actually to this breaking change.

What you propose is exactly what Spring Tool Suite Gradle plugin does for hierarchical Gradle multiprojects using the "Import Gradle Project" wizard. This makes things work, but it's just a compromise.

I mean, having Eclipse *really* handle nested projects would allow to do some interesting things like invoking some commands on the WHOLE project hierarchy at once. The first thing that get into my mind is SCM operations like update or even merge. When you have a multiproject with about 20 subprojects, the hierarchical structure is favourable exactly for this reason and until now we had to struggle with Eclipse to do certain things using tweaks and workarounds.

Maven and Gradle nowadays are great tools and their adoption is continuously increasing. For me, Eclipse lost an occasion to do this resource API change with the 4.x version, but I think that it will never be too late to think about it.

Maybe a new workspace format that is recognized only by the new clients and which is used only on demand?

> While this isn't quite as slick as your proposal, it has the major benefit
> of not breaking existing plugins and provides similar capabilities to your
> prototype. My general feeling is that we allow physically nested projects
> today, but they are difficult to use and often the resulting duplicate
> resource trees are not desirable. I see a lot of room for usability

Actually, the risk to have duplicate resource trees is there once you start to play with resource filters and to have different "roots" that actually are displaced on different branches of your resource tree (think of the case in which the resource filter is "lost" or a plugin does not perfectly respects resource filters, maybe because it runs external commands/scripts that are not resource filter-aware). If the platform could really support nested projects as such, I don't see how I could have duplicate resources along my workspace.
Comment 23 Mauro Molinari CLA 2013-07-18 03:27:05 EDT
(In reply to comment #20)
> While playing around with current UI, I see the "existing project" import
> wizard doesn't even allow you to create/import a project that is physically
> nested in another project. You can still do it, but you have to use File >

Hmmm... strange... Wasn't this one of the "new and noteworthy" feature of Kepler?
See "Import nested projects" at http://download.eclipse.org/eclipse/downloads/drops4/R-4.3-201306052000/news/eclipse-news-part1.html

Or are you rather talking about a different thing?
Comment 24 Mickael Istria CLA 2013-07-18 04:39:15 EDT
@John: The getProject(String projectName) method is the main blocking point for nested folders to happen "natively". Supporting a getProject(IPath projectPath) is the cornerstone of the cleanest implementation of nested projects.

However, I do understand your reticence in changing such a critical API, and I like your proposal of implementing some nested project behaviour using Resource Filters and adapting UI to make it easier to deal with nested projects. This proposal, which is pure API usage and no API change, would already be a major improvement.
So I'm in favor of adopting your proposal, see where it leads and what are its pitfalls; and if we still feel a need to change API, we can do it as another iteration.

However, my knowledge of the Resources UI is very limited, so I don't think I'll be able to provide code for that.


Also, I'm wondering: is it more difficult to deal with nested folders using resource filters, or extending Resource API? As extending resource API seems to be the most sustainable fix, it should probably be the preferred solution IMO.
Comment 25 Mauro Molinari CLA 2013-07-18 05:09:39 EDT
Believe me, changing Eclipse so that it can out-of-the-box handle nested projects using resource filters would be much better than nothing, but it would not be painless. I and my team have been working on this scenario for a couple of years now and there are drawbacks.

First of all, every time you make a structure change (for instance: create a new nested project) you have to:
- "import" the new project into the workspace (i.e.: make Eclipse convert the folder into a project)
- remember to update the resource filters of the parent project
- restart Eclipse in order to make the Subversive plugin recognize the new project "structure" correctly
Apart from the last point, even the automatic handling the first two things would not be so easy, I think.

Then, the SCM chapter: Subversive uses SVN connectors that perform operations on the local filesystem and then applies some actions to refresh itself after an SVN operation has been done. This causes a lot of headaches when nested projects are involved:
- how deep an update operation should go?
- and what about a merge operation?
- and what about a commit operation?
- and what about a revert operation?
In general, what should happen to the child project when you apply an action on its parent project? In a "flat" view, changes should be completely isolated to the parent project only (the hierarchy it's just a "detail" on how projects are organized on disk), but, as I already said, I don't think this is what the user wants/expects when he/she has a project hierarchy and it's not so easy to handle when external scripts/commands are launched (I can think of SVN commands, Ant/Maven/Gradle scripts, etc.). In other words, Eclipse should handle the case in which a physically nested project could receive changes from an action issued on a physically parent project, otherwise the whole thing would feel much like a broken one.

The subject is complex and I've already explained my proposals in bug #392882 for Subversive. But I think that it would be much cleaner and leaner if just Eclipse did support physically nested projects for what they are: in that case, many of these problems would have a "natural" answer rather than a complex one caused by the abuse of resource filters to somewhat handle the hierarchy.
Comment 26 Jörg Thönnes CLA 2013-07-18 05:18:01 EDT
Mauro, w.r.t. to Subversive I put the Alexander Gurov, the main committer and lead for Subversive on copy.

Alexander, may be you could comment on this?
Comment 27 Mauro Molinari CLA 2013-07-18 05:27:10 EDT
(In reply to comment #25)
> Apart from the last point, even the automatic handling the first two things
> would not be so easy, I think.

I'd like to explain better this. Having an action "convert from folder to project" and vice-versa may help here, but it isn't enough. When projects are already properly set up on disk and you want to import them into your workspace, the Import wizard should automatically handle the "reconstruction" of the projects even in the presence of nested projects. But it's not yet enough. I can think of:
- checking out projects from an SCM repository: the project hierarchy may be already set up (with nested projects and resource filters already in place), so the checkout wizard should take this into account
- Team Sets? Are there any impact on them?
- what about the handling of externally generated project structures, with .project and resource filters already in place? Should they be taken automatically by issuing a "Refresh" operation? Right now, when you make a "Refresh" any change is picked up by Eclipse, so I would expect it to automatically recognize nested project once they are found upon a refresh operation

And so on. Both strategies (i.e.: handling real project hierarchies vs. using resource filters and some automations to handle hierarchies) should take a lot of things into account, but the former would make this explicit (and would require a new API support from the consumer plugins, for instance), while the second one would make this implicit, with all the consequences that can be imagined (like subtle cases not correctly handled or whose actual handling strategy could be debatable). 

Put in other words, the "let's use resource filters and things should work" approach has the risk to make a lot of plugins "somewhat work" with project hierarchies, but with the negative feeling of fragility that is hurting Eclipse recently.
Comment 28 Alexander Gurov CLA 2013-07-21 13:30:36 EDT
(In reply to comment #26)
> Alexander, may be you could comment on this?

For the moment I think that each subsystem that may interact with the mentioned case should handle it in the way most fitting the subsystem itself. For example, a Team provider should allow user to work with the nested projects the same way they work with the root project. If this condition is reached, then the user who is working with such a project structure will just use Eclipse IDE's standard tools to manage the resource tree presentation (this is where I may expect some changes in Eclipse IDE, such as standard filters, allowing user to hide a nested project's content in the root project, for example).
One way or another at some point in time, when you work with resources tree, you'll be forced to handle the nested projects structure. For example, sharing a nested project, but not the root one: is it a user's decision or just a mistake? Etc.

Refreshing an overlapping resources is not such a hard task in the end (since you can always check with the existing API if the resource has any counterparts in different projects or not). The logic behind a user's decision is what worries me the most, since the ideal solution matching all the possible requirements just does not exists, I think each subsystem should handle its own aspect of the whole problem, for example:
- Team providers should correctly handle refreshing resources, provide some services for sharing, checking out projects
- Navigator, Package Explorer and any other view with resources tree should provide a filter, that allows to hide nested project's content in the root project, should refresh all the resource conterparts when F5 refresh action is requested
- "Import projects" wizard should allow importing nested projects without any additional actions on the user's side
- etc.

P.S.
What I think definitely should be changed in Eclipse Platform regarding this issue:
1) when a team provider is asked by the file modification validator about what to do when user tries to change a read-only file, the resource locking context should contain all the counterparts for the read-only file in a nested project, otherwise the only way to properly refresh an overlapped resource is to reschedule a delayed refresh operation for all the counterparts under different resource locking rules (yes, it works, but looks kind of lame and stupid, and works asynchronously, which may lead to a negative user expirience).
2) The Team provider should not be the one to handle resource tree presentation filters, because we can work with nested projects, but without any Team providers.
3) And then there is a possible improvement (if I'm not wrong, it could be done without any API changes): since locking and refreshing an overlapped resource are stadard tasks, it would be great to do it for all the counterparts automatically, i.e. lock all the counterparts when a single one is locked, refresh all the counterparts when a single one is refreshed, since all of them are representing the same actual resource.
Comment 29 Mickael Istria CLA 2013-07-22 04:55:26 EDT
From my end-user POV, I don't think the feature of nested projects depends on any constraint of the Team Provider, so I think nested folders could/should be implemented independently of how Team Providers are currently working.
Let's say we go for supporting the use-case shown in the video I posted earlier (which is to have a root project org.eclipse.gmf-tooling and open plugins/org.eclipse.gmf.tooling.map as a nested project), or that we think about a Maven tree with root project being the top-level element and nested project being some individual modules. With these use-cases, it is totally fine to have the Team Provider connected only to the root project and handle nested projects just like simple folders (which they actually are from the SCM POV).

The nested SCM Trees (for example a git project inside a Git project) is an issue that is specific to Team Providers and that can already happens with Folders.
Support of nested project in Resources can be done independently of Team Providers.
Support of nested SCM trees in Team provider can be done independently of Nested Projects.
Comment 30 Robin Stocker CLA 2013-07-22 07:03:19 EDT
In the Git team provider (EGit), we support nested projects with and without resource filters (see bug 358901, where support for resource filters was implemented). There are some kinks, such as decorations on a parent project including the changes from nested projects, even if they're filtered. Also, some operations work on the file system level instead of traversing the resource tree.

IMO the infrastructure is mostly there, but the UI has to be improved to make the support more visible to users. +1 for John's suggestions in comment 19.
Comment 31 Mickael Istria CLA 2013-08-22 05:17:54 EDT
Do you think there is any chance to get some support for nested project included as part of Luna? If we want to do so, it would make sense to get this implemented/integrated ASAP so we'll have some time to debug it and let other projects leverage it.
IMO, the patch I suggested (with some necessary fix/improvements I guess) could make a first increment.
Comment 32 John Arthorne CLA 2013-08-27 14:27:38 EDT
(In reply to comment #31)
> Do you think there is any chance to get some support for nested project
> included as part of Luna? If we want to do so, it would make sense to get
> this implemented/integrated ASAP so we'll have some time to debug it and let
> other projects leverage it.
> IMO, the patch I suggested (with some necessary fix/improvements I guess)
> could make a first increment.

I believe there are essentially no active committers on this component today. If someone steps forward to do the work, we can probably find a committer to review, or make the contributor a committer quickly enough so they can develop and maintain it.
Comment 33 Mickael Istria CLA 2013-08-28 04:28:04 EDT
(In reply to comment #32)
> > IMO, the patch I suggested (with some necessary fix/improvements I guess)
> > could make a first increment.
> 
> I believe there are essentially no active committers on this component
> today. If someone steps forward to do the work, we can probably find a
> committer to review, or make the contributor a committer quickly enough so
> they can develop and maintain it.

I'm ready to do the work by extending resources API, and the patch I submitted (https://git.eclipse.org/r/#/c/14108/ ) is actually the first piece of work.
However, the code for Resources code is not trivial (which is fair since Resource API in Eclipse has a lot of advanced features), so I don't feel confident yet on getting some patches merged without someone experimented with Resource implementation to review it.
Comment 34 Mickael Istria CLA 2013-09-13 14:44:10 EDT
Evaluating how much trouble it would be to adopt the suggested patch and change behaviour, relying only the Javadoc as specification.

Here is the only thing I've seen which specifies the relationship between IWorkspaceRoot and IProject:
* getProject(String name) :  Returns a handle to the project resource with the given name which is a child of this root.
* getProjects() :   Returns the collection of projects which exist under this root.

"a child of this root" and "under this root" don't make it clear whether this applies to sub-children or not, and current implementation doesn't take a decision on this.
In the end, it seems like the implementation of nested folder is just about refining the specification of those 2 methods to make clear whether they handle all projects or not.

Nothing specified that project.getParent() has to be the IWorkspaceRoot, (and nothing specify that all children of workspaceroot have to be projects).
Comment 35 Jens Ohlsson CLA 2014-01-27 03:50:56 EST
Any news on this one? I´m getting more and more sick of the flat project layout as the number of submodules grows in my (maven) projects.

I would be really useful if this means eclipse would accept importing projects with same name, as long as their path differs.
Comment 36 Mickael Istria CLA 2014-02-07 08:18:51 EST
As it seems like it is a pure UI issue and that it could be implemented without requiring any change in Resource API nor implementation, shouldn't this issue be moved to Platform/UI ?
Comment 37 Igor Fedorenko CLA 2014-02-07 09:00:28 EST
As far as Maven support is concerned, I do not believe m2e needs anything from Platform/Resources to implement proper support for Maven multi-module project layout. I believe the changes will only affect JDT (and m2e, if course). I have pretty good idea what needs to be done, just lack the time/resources to do it.
Comment 38 Sebastian Dietrich CLA 2014-02-07 12:46:03 EST
(In reply to Igor Fedorenko from comment #37)

So we have a 6 year old bug with 33 votes that blocks a number of bugs, some of them >10 years old. And we have someone who thinks he is capable of fixing the bug, just lacking the time/resources to do it.

Maybe we should start collecting money so the time/resources is no longer the blocker for fixing this bug...
I am e.g. willing to pay € 100,- personally.
Comment 39 Neale Upstone CLA 2014-02-07 14:34:05 EST
+100 euros from me

BTW. I think what we're doing here is voting for financial reform of the Eclipse foundation... ;-)

In my understanding, much of the direction is influenced by full-time staff of IBM and others.  I think the foundation needs funding and employees to act for the community as a whole.
Comment 40 Igor Fedorenko CLA 2014-02-08 09:14:43 EST
I appreciate the offer, but between jdt and m2e there is at least several months worth of work, so I am afraid 200 euros isn't quite enough. If, however, you work for a company interested to see this implemented, feel free to contact me directly and we see if there is a way to make this happen.
Comment 41 Mickael Istria CLA 2014-02-10 03:29:34 EST
FYI, I'm working on a creating some filters to nest projects in navigator view. It's not at all about "physical" nesting and it's a pure UI make-up so some concerns raised in previous comment will remain true, but it would be an interesting and easier step forward. Cf bug 427768
Comment 42 Mickael Istria CLA 2015-03-24 04:56:47 EDT
As from Mars M5, the Project Explorer provided by Platform/UI allows to set "Project View > Hierarchical" which allows to show projects as nested.
This does not change the internal resource model, it's just a view customization that relies on projects location to hide/show projects at alternative location in the Project Explorer tree.
https://www.eclipse.org/eclipse/news/4.5/M5/#nested-projects

We can keep this ticket open for brainstorming about the resource model, but IMO, the resource model can remain totally internal and hidden to the user, and all this "nested projects" story can be resolved by adding this concept in UI (which is much easier to do than hacking the resource model).
Comment 43 Martin Oberhuber CLA 2015-03-24 05:15:33 EDT
Hi Mickael,

Assume the following physical structure:

   foo
     .project
     bar
       .project
       barfile.txt

Will barfile.txt show up 2 times in the project explorer (because it's under foo and under bar) ?
Comment 44 Martin Oberhuber CLA 2015-03-24 05:17:55 EDT
> Will barfile.txt show up 2 times in the project explorer (because it's under
> foo and under bar) ?

I mean, if a text search for example finds a match in barfile.txt , will barfile.txt be displayed 2 times in the search results ? Or similar, if there is a compile error or task tag in barfile.txt ... will 2 items be displayed in the Problems view or the Tasks view respectively ?
Comment 45 Mickael Istria CLA 2015-03-24 05:20:19 EDT
(In reply to Martin Oberhuber from comment #44)
> > Will barfile.txt show up 2 times in the project explorer (because it's under
> > foo and under bar) ?
> 
> I mean, if a text search for example finds a match in barfile.txt , will
> barfile.txt be displayed 2 times in the search results ? Or similar, if
> there is a compile error or task tag in barfile.txt ... will 2 items be
> displayed in the Problems view or the Tasks view respectively ?

I suggest you give it a try, but the answer is yes, it will show up multiple times in viewers that don't group resources by location.
Other part of UI responsible for showing resources have to adapt too.
Comment 46 Max Rydahl Andersen CLA 2015-03-24 06:25:34 EDT
Searching in particular should be able to know its hitting the same physical location and filter those out. Problem comes to know in which context/project the file should be opened.
Comment 47 Martin Oberhuber CLA 2015-03-24 06:29:44 EDT
Well, from the point of view of all existing Eclipse plug-ins we still have 2 independent resources (which happen to map to the same physical file). It looks like most tools handle this "kinda OK" but I thought that the discussions on this bug were about finding a solution for that problem.
Comment 48 Pedro Silva CLA 2015-03-24 10:12:13 EDT
This is especially problematic for Java classes.
What's the point on opening the Java class file that's not on the faceted Java Project? You won't be able to jump through types, perform refactoring and so on... It's just pollution...

In my opinion there are two options:

1- Eclipse or each plugin adapts to provide in the open resource or the open type boxed a way to group "mounted" projects from the others. And by doing this hitting the wrong file would be minimized.

2- Eclipse or each plugins adapt to implement a set of rules that filter out resources and Types:
 1) are there multiple resources?
 1.1) if so see which comes from a faceted project
  1.1.1) if there is only one project show that one
  1.1.2) if there are multiple, provide them all
 1.2) if not show the one from the nearest project in the tree
Comment 49 Max Rydahl Andersen CLA 2015-03-24 13:29:43 EDT
"1.2) if not show the one from the nearest project in the tree"

I can't think of a case where this would not always be the right one ?
Comment 50 Pedro Silva CLA 2015-03-24 13:48:12 EDT
What I'm assuming is that, that would mean that the resource you are looking for is on a project that is not imported or on a plain project folder.

The suggestion is that if you are on a nested project you'd go up in the tree and show the first match, if you are in the root it would be the same but downwards, right?
Comment 51 Mickael Istria CLA 2015-03-25 02:05:38 EDT
Guys, as Martin explained in previous comment, let's keep this bug open to keep hope in the fact that nested projects can be handled in the Resource Model.
If you have any comment about the nested project view in Project Explorer, please come discuss that on the platform-ui-dev mailing-list or open bugs against Platform/UI.
Comment 52 Igor Fedorenko CLA 2015-03-28 08:23:43 EDT
*** Bug 463355 has been marked as a duplicate of this bug. ***
Comment 53 Tushar Kapila CLA 2015-04-01 03:54:17 EDT
(In reply to Igor Fedorenko from comment #52)
> *** Bug 463355 has been marked as a duplicate of this bug. ***

project/
  pom.xml
  README.MD
  setup.sh
  moduleA/
    pom.xml
    src/main/java/
    src/test/java/
  moduleB/
    pom.xml
    src/main/java/
    src/test/java/

At the very least :
1. importing a maven/gradle project. 
2.The sub projects become high level projects
3. able to delete the sub projects from the main entry (but they are still indepndant projects)
4. on the main project able to edit the main pom, read me etc

Now I dont import the main project as then classes are duplicated etc. So maybe a check box - import main but not children under it. (But still import children if selected as top level independent projects as we can now.
Comment 54 Mickael Istria CLA 2015-04-01 04:06:00 EDT
(In reply to Tushar Kapila from comment #53)
> Now I dont import the main project as then classes are duplicated etc. So
> maybe a check box - import main but not children under it. (But still import
> children if selected as top level independent projects as we can now.

For a better import story, you might be interested in giving a try to this ongoing effort: https://wiki.eclipse.org/E4/UI/Smart_Import , which provide the checkbox you mention.
About nested view, there is something ready in Mars M5 and later, that you can already try: https://www.eclipse.org/eclipse/news/4.5/M5/#nested-projects
Combining both should give you something that fits your use-case better.

Also, what you mention with duplicated classes actually doesn't happen for JDT, since JDT would only see classes of your sub-modules. So using "Open Type" (Ctrl+Shift+T) would show those classes only once.
However, it's true that resources (Ctrl+Shift+R, Search...) would still appear twice.
Comment 55 Martin Oberhuber CLA 2015-04-01 16:19:45 EDT
(In reply to Mickael Istria from comment #54)
> However, it's true that resources (Ctrl+Shift+R, Search...) would still
> appear twice.

Eclipse supports resource filters. Has anybody thought about providing a resource filter that would filter-out any folder(s) that match the project location of a physically nested project ?
Comment 56 Mauro Molinari CLA 2015-04-01 16:55:50 EDT
(In reply to Martin Oberhuber from comment #55)
> Eclipse supports resource filters. Has anybody thought about providing a
> resource filter that would filter-out any folder(s) that match the project
> location of a physically nested project ?

This is exactly what the Gradle plugin for Eclipse from Spring Tool Suite does: it configures nested projects as independent ones and adds resource filters to filter them from the root project. It works quite good, but the fact that nested projects aren't natively supported by Eclipse can cheat some other plugins (like Subversive, for instance) in some circumstances.
If Eclipse could support out-of-a-box the configuration of nested projects with features it already has (like resource filters), it would be very useful. If it could support nested projects natively, so that all the plugins could then be appropriately adapted (sooner or later), it would be even better IMHO.
Comment 57 Brian de Alwis CLA 2016-03-21 14:00:10 EDT
(In reply to Martin Oberhuber from comment #55)
> (In reply to Mickael Istria from comment #54)
> > However, it's true that resources (Ctrl+Shift+R, Search...) would still
> > appear twice.
> 
> Eclipse supports resource filters. Has anybody thought about providing a
> resource filter that would filter-out any folder(s) that match the project
> location of a physically nested project ?

I was just looking for exactly this and so opened bug 490110.