Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: FW: [platform-core-dev] RE: [cdt-dev] First gotchawith add/excludechildren of FFS

I'm for separating this meeting from CDT BOF. I'd like to spend CDT BOF talking about other issues (binding resolution, context assist, etc).

-sergey

On Thu, Mar 13, 2008 at 10:06 AM, Schaefer, Doug <Doug.Schaefer@xxxxxxxxxxxxx> wrote:
Well I'll be there and I want to talk about flexible file systems. Szymon can you make this time?

We can also use lunch times, I guess.

Doug.

-----Original Message-----
From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Chris Recoskie
Sent: Thursday, March 13, 2008 12:55 PM
To: CDT General developers list.
Subject: RE: [platform-core-dev] RE: [cdt-dev] First gotchawith add/excludechildren of FFS

I guess it depends?

We really have no idea who is showing up to the BOF or what they want to talk about.  Do we know if Szymon can make that time?

===========================

Chris Recoskie
Team Lead, IBM CDT Team
IBM Toronto
http://www.eclipse.org/cdt




            "Schaefer, Doug"
            <Doug.Schaefer@wi
            ndriver.com>                                               To
            Sent by:                  "CDT General developers list."
            cdt-dev-bounces@e         <cdt-dev@xxxxxxxxxxx>
            clipse.org                                                 cc

                                                                  Subject
            03/13/2008 12:50          RE: [platform-core-dev] RE:
            PM                        [cdt-dev] First gotcha with
                                      add/excludechildren of FFS

            Please respond to
              "CDT General
            developers list."
            <cdt-dev@eclipse.
                  org>






Did we just want to do this at the CDT BOF? I don't see much room in the schedule. The CDT BOF is 8:45 on Wed. Thoughts?

Doug

-----Original Message-----
From: platform-core-dev-bounces@xxxxxxxxxxx
[mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of Chris Recoskie
Sent: Wednesday, March 12, 2008 9:41 AM
To: CDT General developers list.
Cc: platform-core-dev@xxxxxxxxxxx
Subject: [platform-core-dev] RE: [cdt-dev] First gotcha with add/excludechildren of FFS

Please count me in for such a meeting.

===========================

Chris Recoskie
Team Lead, IBM CDT Team
IBM Toronto
http://www.eclipse.org/cdt




            "Schaefer, Doug"
            <Doug.Schaefer@wi
            ndriver.com>                                               To
            Sent by:                  "CDT General developers list."
            cdt-dev-bounces@e         <cdt-dev@xxxxxxxxxxx>,
            clipse.org                <platform-core-dev@xxxxxxxxxxx>
                                                                       cc

            03/10/2008 10:27                                      Subject
            AM                        RE: [cdt-dev] First gotcha with
                                      add/exclude children of FFS

            Please respond to
              "CDT General
            developers list."
            <cdt-dev@eclipse.
                  org>






Copying the platform-core-dev folks too. Is there someone from the Platform who could attend a meeting at EclipseCon about flexible file systems. John A, will you be there? John is Mr. EFS and we can use his guidance.

Thanks,
Doug.

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Ken Ryall
Sent: Saturday, March 08, 2008 5:08 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] First gotcha with add/exclude children of FFS

Doug,

Can we get together at EclipseCon to discuss this issue specifically? Do you know the right platform people to invite to the meeting? We really need to piece together a plan.

I?m sure you have enough to do so if you can tell me who to recruit I can help organize the meeting.

Thanks - Ken

From: "ext Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
Reply-To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Date: Wed, 20 Feb 2008 18:44:57 -0800
To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Subject: RE: [cdt-dev] First gotcha with add/exclude children of FFS

So, you know. The more I think about what you guys are saying, I'm realizing that the EFS solution probably is the right one. The objective should be to turn the IResource tree into a logical project view and to remove all notions that it represents physical file layout. That unfortunately starts with the .project and .cproject files, but I think there are tricks we can do there. The .settings may be harder but let me sleep on that.

At any rate, this has piqued my interest again and I'll work on reviving it and see where it goes. I'll try to get a prototype working by EclipseCon so we can talk about it more concretely.

Cheers,
Doug

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Brunauer, Walter
Sent: Monday, February 18, 2008 8:53 AM
To: CDT General developers list.
Subject: RE: [cdt-dev] First gotcha with add/exclude children of FFS

Hi Warren,

well, the confusion my origin from the different meanings of what project setup is: for me, project setup is not equal to build setup. I.e., on our projects, the build setup is an independent step from the project setup. We intentionally separated this to overcome all kind of issues you obviously experienced as well. And this is how we see it:

1. Create a project at the desired location (everything beneath this root location is part of the project, but it can be an empty project just as well with linked resources added to it later). By default, the build setup is identical to the project content (there is one build target, linking/archiving everything together).

2. If (a) specific build setup(s) are needed, it is possible to specify as many build targets with arbitrary contents as desired. This approach separates the physical file system layout from logical build layouts, and it even works beyond project boundaries. IOW, no matter from where source files are pulled in (the same projects, nested projects, outer projects, sibling projects), one is able to specify any build setup exactly are needed, as long as all source files are known to Eclipse (as resources).

HTH,

Walter



     From: cdt-dev-bounces@xxxxxxxxxxx  [
     mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
     Warren.Paul@xxxxxxxxx
     Sent: Freitag, 15. Februar 2008  14:51
     To: cdt-dev@xxxxxxxxxxx
     Subject: RE: [cdt-dev] First  gotcha with add/exclude children of FFS



     Hi Walter,



     I forgot all about the absolute paths issue with linked  resources.
     I'll update the wiki.



     I'm a bit confused about your comment about this not  being a project
     setup issue.  We have our own builder as well, and it  will happily
     build whatever the build description says, whether those files  are
     under the project root or not.  We even have our own project explorer
     view which shows a logical representation of the project rather than
     the  physical file system layout.  But we still run into a lot of
     issues when  files are not under the project root - that is, when you
     can't get an IFile  for them.



     We have a wide range of user types from small application  developers
     to large system developers.  In many cases, a users code base
     consists of hundreds of directories with thousands of source files.
     In  such a source base, there are many hundreds of build artifacts
     and almost as  many "logical projects".  It is a huge problem for
     these users to be able  to create projects currently.  They will
     typically have a few projects  going at a time, but many times the
     natural project root for all of them will  be the same.  We've found
     ways to work around some of the other issues,  but not this one.  It
     sounds like perhaps you guys have found a  way.  Could you elaborate
     on how you setup your  projects?



     Thanks,
     Warren



     From: cdt-dev-bounces@xxxxxxxxxxx  [
     mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of ext Brunauer,
     Walter
     Sent: Friday, February 15, 2008 1:56 AM
     To: CDT  General developers list.
     Subject: RE: [cdt-dev] First gotcha with  add/exclude children of FFS



     Hi Warren,



     FWIW, you did not mention anything about linked  resources and
     absolute paths these persist in the .project file by default.  Again
     a big issue around linked resources in combination with sharing
     project  within a team (even without team support), and one more
     reason why they appear  to be so cumbersome to handle. To me it seems
     many times one has  to unsell linked resources to users: Whereas
     linked resources  are (kind of!) nice for evaluation purposes
     (because, yes, in this case  you might not want to pollute your
     sources), as soon as you start serious  development, you run into all
     kind of troubles. The hurdle to get everything  right from the
     beginning is overwhelming for novices (e.g. its not possible to
     change a linked resource to use a variable later). Sorry, I don't
     know how to  add this to the Wiki page...



     Having said that, the scenario you describe is  really about having
     the flexibility around build and indexer setup, not around  project
     setup, IMO.



     It's rather classic: users have common  code they want to reuse in
     multiple applications - so they create one or  a set of libraries out
     of it, within one or a set of projects. Of course,  indexing should
     be able to handle only code going into these libraries, and
     optionally ignore the rest. Then, they create their application
     projects, which  use the binary artifacts of the library project(s).
     Now it would be great if  they would have automatic support for
     application linkage specification, i.e.  some nice wizard or UI
     allowing to select the library binaries of other  projects to be
     linked in, without the need to specify it manually in the  linker
     options. And probably also desired: during application code
     development, the public API's of all used library projects should be
     the only  thing they see WRT code completion, etc. I guess, some UI
     would be needed for  this as well.



     And now think of all developers in the world.  Wouldn't it be great
     to give as many of them the freedom to choose how to  achieve this?
     Either everything in one project, or one project per build  artifact,
     or one project per module/application/product, or with nested
     projects... its possible. Our commercial IDE based on CDT supports
     all this,  and we did not have to provide some EFS or work with
     linked resources.  Well, we had to override the build system, and
     this is IMO the place to solve  this in CDT as well.




     Again, I don't see anything specific to project  setup. The issue
     around having the source tree polluted with project files - I  don't
     think this is the big thing. I would not leave the Eclipse path in
     this  area at all and allow to separate the project file from the
     project location.  Its a very general paradigm of Eclipse, and I am
     pretty sure doing everything  differently will generate lots and lots
     of problems in all kind of areas  (probably much more than you
     already identified), unless you make it a new  Eclipse way
     (add/change this in the platform, not in CDT, that  is).



     Just my 2 or 3 cents again,



     Walter




           From: cdt-dev-bounces@xxxxxxxxxxx  [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
           Warren.Paul@xxxxxxxxx
           Sent: Donnerstag, 14. Februar 2008  23:35
           To: cdt-dev@xxxxxxxxxxx
           Subject: RE: [cdt-dev]  First gotcha with add/exclude children
           of FFS



           I've updated the Wiki page
           http://wiki.eclipse.org/CDT:Flexible_Project_Structure with
           some more thoughts on the issue.  It would be great to get
           feedback  from other CDT users - both those shipping C/C++
           IDE's and end users.   You'll see that I'm not convinced that
           the linked resources route is a  viable option.  Maybe we can
           get the platform team involved in the  discussion to help find
           the best route forward.



           Thanks,
           Warren





           From: cdt-dev-bounces@xxxxxxxxxxx  [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of ext Schaefer,
           Doug
           Sent: Friday, January 25, 2008 11:15 AM
           To: CDT  General developers list.
           Subject: RE: [cdt-dev] First gotcha with  add/exclude children
           of FFS



           I guess what my investigation has shown me that the EFS
           solution and linked resources are pretty much identical. I
           really  noticed this when trying to figure out how to persist
           the adds and found  myself wishing I could add that to the
           .project file just like linked  resource are. And they are....



           I think all the issues that we have with linked resources would
           be equally as bad with the EFS solution,  possibly worse
           because the EFS adds are hidden. The CVS one is a great
           example. I really doubt CVS would work with the EFS solution
           either. And I  don't want us to think EFS would be better since
           it's not in the platform  where we'll have a battle getting
           changes. Any platform changes required to  make linked resource
           work correctly would also need to be done for  EFS.



           So my hope is to save the effort at implementing  the
           add/remove functionality since I believe that's already there
           with  linked/hidden resources. We can then focus on making
           linked resources  work where we need them and improving the
           workflows. But this  really needs to start now.



           So, Warren, you've somewhat started a list of  workflows that
           we'd like to support with this solution. This is a great  place
           to start. I've created a Wiki page to start capturing these.
           Please  feel free to add more information, especially to the
           workflow section. When  we have that we may get a better idea
           of which of the two solutions will  work best.



              http://wiki.eclipse.org/CDT:Flexible_Project_Structure



           Thanks,

           Doug





           From: cdt-dev-bounces@xxxxxxxxxxx  [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
           Warren.Paul@xxxxxxxxx
           Sent: Friday, January 25, 2008 11:22  AM
           To: cdt-dev@xxxxxxxxxxx
           Subject: RE: [cdt-dev] First  gotcha with add/exclude children
           of FFS



           We've been working on Eclipse/CDT based products for  about
           three years now.  I'm sorry to say that the project model is
           still not satisfactory for our purposes.  We've tried many
           angles, but  are still stuck with some pretty serious
           limitations.  I've volunteered  to investigate the EFS route to
           see if it will help at all.  Based on  this thread I'm assuming
           it won't.



           Let me give you a brief overview of how our users work,  and
           then discuss the problem we've run into.  I don't think any of
           this  is specific to our users BTW.



           Most of our users have existing code bases.  They  simply want
           to "import" it into the IDE.  Others will create new  projects
           from our templates.  The new projects are created in the
           workspace.  Imported projects could be anywhere in the file
           system.  Often times they will import several projects from the
           same  source tree.  This is where our biggest problem is.
           Let's say the  source base looks like this:



           C:\MyProjects\Project1\...


           C:\MyProjects\Project2\...


           C:\MyProjects\Common\...



           Because both projects  share code in the Common directory, the
           logical root project directory for  both Project1 and Project 2
           is C:\MyProjects\.  But in Eclipse you  can't have two projects
           with the same root.  This is where the .project  and .cproject
           files are created.  So currently our users would import
           Project1 with the natural root (C:\MyProjects\), but Project2
           has to be  rooted at C:\MyProjects\Project2\.  This means that
           any source/headers  from the common directory are not under
           Project2.  This means those  files are not in the project
           explorer for that project, are not indexed,  etc..  We logged
           this against the platform -
           https://bugs.eclipse.org/bugs/show_bug.cgi?id=78438.
           Basically if you put the .project file anywhere, but have a
           project root  attribute, this would cease to be a  problem.



           Our first product actually always created the  .project in the
           workspace, and for imported projects, would create links to
           files and folders.  We ran into so many issues with this that
           we had to  change the model.  I don't recall all of the issues,
           but here's a list  of some:



           - Version control simply didn't work at  all



           - You can't make file system changes with  links.  For example,
           if you want to rename a file or folder, or move a  file around,
           you can't do this with linked resources.  It only changes  the
           link itself, not the underlying  resource.



           - Creating new resources in a project with links is  confusing
           at best.  Let's say you have a project with a linked folder
           and file at the root.  If you create a new file or folder at
           the root,  it is created in the workspace, not where the other
           folder/file are in the  file system.  But if you create a new
           file under the linked  folder, it gets created where you'd
           expect.



           - The location of the .project/.cproject files are
           problematic.  Some users will want to keep these in version
           control,  while others won't.  Those that do want them created
           in the source  tree, but those that don't want them elsewhere,
           like the workspace.  I  forget now why this was a problem with
           linked resources, but there was  something weird going on
           there.



           I suppose it's worth noting that the last time we  really
           looked at this was in Eclipse 3.2, so some of this may have
           been  fixed by now.  But I doubt it.  In general linked
           resources are  second class citizens.  Some IResource API's
           just don't work for linked  resources.  Just search for
           references to IResource#isLinked for  "special handling".  I
           suspect that we'll run into similar issues with  EFS though -
           see getLocation vs  getLocationURI.



           We also have the same issue that Doug is trying to  address
           (hiding some branches in a source tree).  This is much less of
           an issue for us though.  You can already reduce the scope of
           the  indexer and the build.  The only real issue for us is for
           a very large  source tree, you're going to get IResource's for
           everything, which slows  things down quite a bit.  There is
           actually somewhat of a problem in  reducing the indexer scope -
           see https://bugs.eclipse.org/bugs/show_bug.cgi?id=178159.



           The hidden attribute addition sounds promising for  hiding
           resources under the project root, but doesn't really do
           anything to  add flexibility to the contents of a project.  EFS
           sounds like it would  though.  What I mean by that is, having
           resources under a project that  are real resources, not linked,
           but that don't live under the project root  in the file system.
           I've just started looking into EFS, so maybe it's  a bit of
           wishful thinking at this point, but I'm hoping we could create
           a  project anywhere, and when we create it we pass the URI
           location from our  own EFS.  Then when asked for the children,
           we could return URI's for  files from anywhere in the file
           system, or on other machines even.   This would seem to hold
           the potential for working around the issues listed  above.
           We'd basically have an EFS map from what we want under a
           project to the actual file  system.



           So hopefully some of the experts can chime in  here.  Is my
           hope for EFS unrealistic?  Is there a different  approach we
           should look at?



           Thanks,
           Warren





           From: cdt-dev-bounces@xxxxxxxxxxx  [
           mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of ext Brunauer,
           Walter
           Sent: Friday, January 25, 2008 1:47 AM
           To: CDT  General developers list.
           Subject: RE: [cdt-dev] First gotcha with  add/exclude children
           of FFS



           Hi,



           after reading this rather long thread, I'll  decided to throw
           in my personal opinion.



           I consider this approach to work against one of  the most
           general Eclipse platform paradigms, where a project is defined
           to be a root directory and everything in it. IMO, the more
           workarounds are  introduced against this paradigm, the more
           problems will be faced, and the  more incompatibilities (or at
           least unawarenesses)  created.



           Isn't the whole problem you try to solve here  rather about
           what files should go into the build (and probably into the
           indexer) than what files are part of a project? I understand
           that CDT has no  separation of what a project and what the
           build input is (well, IIRC one can  exclude specific files from
           the build, but in general, the project content  defines the
           build input, right?).



           In our commercial IDE, we separated this. This not  only
           introduced much more powerful build setup capabilities in
           general, but  especially enabled users to setup build artifacts
           with arbitrary  contents (think of sources being compilable
           with different compiler flags  for different build artifacts,
           build input exclusion patterns, build input  from all over the
           workspace, multiple build artifacts within the same  project,
           reusable build artifacts accross project boundaries, etc.,
           etc.,  etc.). BTW, we call this build system flexible managed
           build - because  that's what it is:-)



           Of course, one can setup CDT projects as of today  to exactly
           contain what is desired (with the help of linked resources).
           However, I find linked resources to be cumbersome and error
           prone, though  many of our customers start out with them during
           evaluation as well, mostly  because they are looking for a way
           to achieve what they did in the past with  other non-Eclipse
           based IDEs, but sooner or later I know of lots of  them
           realizing its much easier to use the features of our flexible
           build  system instead, especially if projects need to be shared
           in a team.  And now think of virtual file systems, the
           potential complexity of  these, hidden assumptions,
           restrictions, etc. Sounds worse than linked  resources to me.



           I guess, the point I am trying to make is:  whatever you decide
           to do, make it understandable  and transparent (and of course
           as simple as possible to use) for  the user.



           As said, just my 2 cents,



           Walter




                 From: cdt-dev-bounces@xxxxxxxxxxx  [
                 mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
                 Schaefer,  Doug
                 Sent: Donnerstag, 24. Jänner 2008 23:17
                 To: CDT  General developers list.
                 Subject: RE: [cdt-dev] First gotcha  with add/exclude
                 children of FFS



                 Jogging through the code, it really looks like the
                 HIDDEN feature is what I was looking for. What I haven't
                 found yet is UI  to make a resource hidden or a navigator
                 filter to show hidden resources  (in case you want to
                 bring them back). Is this planned?



                 Assuming we have the core features available to link  in
                 and hide resource, I think we still have workflow issues
                 that need to  be addressed. I like Ken's idea of a file
                 that controls the  linking/hiding. We could have an
                 import/export mechanism for setting up  projects based on
                 this file. A nice wizard for creating the file would
                 also be good, similar to the way the way the export file
                 system wizard  works.



                 Given this, we may be further along than we thought
                 (BTW, the hidden stuff seems to have been added in 3.4
                 M4).



                 Cheers,

                 Doug



                 From: cdt-dev-bounces@xxxxxxxxxxx  [
                 mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
                 Schaefer,  Doug
                 Sent: Thursday, January 24, 2008 2:51 PM
                 To: CDT  General developers list.
                 Subject: RE: [cdt-dev] First gotcha  with add/exclude
                 children of FFS



                 Thanks Michael/Szymon,



                 Is there a bug describing the isHidden  feature?


                 Doug



                 From: cdt-dev-bounces@xxxxxxxxxxx  [
                 mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of Michael
                 Valenta
                 Sent: Thursday, January 24, 2008 11:37 AM
                 To:  cdt-dev@xxxxxxxxxxx
                 Subject: RE: [cdt-dev] First gotcha with  add/exclude
                 children of FFS


                 Doug et al,

                 Szymon is really the person you want to bug  on this but
                 I'll throw in my 2 cents ;-)  First, I have to say that a
                 solution at the IResource level (e.g. using linked
                 resources and the new  hidden folder support) is
                 infinitely better from a repository provider  perspective
                 than an EFS based solution.  You may not get all the Team
                 support you want at the IResource level but a solution at
                 the EFS level  would certainly break the existing CVS
                 client since the CVS client isn't  EFS aware to any great
                 extent. For instance, if you tried to hide a folder
                 using EFS, the CVS client would probably try and recreate
                 it the next time  you performed a Team>Update. It is also
                 important to note that the  Platform does not provide all
                 the hooks required by repository providers  and I know of
                 at least one provider that has resorted to using it's own
                 EFS implementation under projects that are mapped to that
                 provider to get  the capabilities it requires. I think it
                 is important that tooling in  Eclipse stick to using the
                 IResource layer as the layer they operate on  and let the
                 repository provider (or any other tooling whose
                 responsibility  it is to manage the available files)
                 control the underlying file system.  If there are
                 shortcomings or enhancements required then you should
                 push to  get them in at the IResource level.

                 As for the current state of  Team support for linked
                 resources, I think the best approach is to  enumerate
                 some specific scenarios of how you see linked resources
                 and  exclusions working with descriptions of what you
                 need to do today to get  Team support and what you would
                 like to see. It is also important to know  if you expect
                 all the links to come from the same repository (or at
                 least  repository type) or whether a project could
                 contain content from different  repository types
                 (obviously the later would be more difficult to
                 accommodate than the former).

                 Hope this  helps,
                 Michael


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev


Back to the top