RE: [platform-core-dev] RE: [cdt-dev] First gotcha with add/excludechild

[John Arthorne <John_Arthorne@xxxxxxxxxx>]

Hi Doug,

Unfortunately as Szymon said I won't
be able to make it to EclipseCON this year.  Szymon is the active
developer in this area so he is the right person to talk to.  Can
you let this list know if/when you've settled on a date for meeting? I
may be able to track someone down who's attending who can give some historical
perspective on things.

John

"Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
Sent by: platform-core-dev-bounces@xxxxxxxxxxx

03/10/2008 11:18 AM

Please respond to
"Eclipse Platform Core component developers list."        <platform-core-dev@xxxxxxxxxxx>

To	"Eclipse Platform Core component developers list." <platform-core-dev@xxxxxxxxxxx>
cc	"CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Subject	RE: [platform-core-dev] RE: [cdt-dev] First gotcha with        add/excludechildren of FFS

Thanks Szymon. That's too bad about John. But I'm
glad you'll be there. We'll talk on the cdt-dev list to see when everyone
will be free. I'll make sure the messages are forwarded to the platform-core
list.

Cheers,
Doug.

-----Original Message-----
From: platform-core-dev-bounces@xxxxxxxxxxx [mailto:platform-core-dev-bounces@xxxxxxxxxxx]
On Behalf Of Szymon Brandys
Sent: Monday, March 10, 2008 10:43 AM
To: Eclipse Platform Core component developers list.
Subject: Re: [platform-core-dev] RE: [cdt-dev] First gotcha with add/excludechildren
of FFS

Hi. As far as I know John is not coming. I will try to attend the meeting.
When are you going to make it?

--
Szymon Brandys



                    
                     
                     
          
             "Schaefer, Doug"  
                     
                     
             <Doug.Schaefer@wi  
                     
                    
             ndriver.com>    
                     
                    To
             Sent by:      
           "CDT General developers list."
     
             platform-core-dev    
    <cdt-dev@xxxxxxxxxxx>,        
     
             -bounces@eclipse.    
    <platform-core-dev@xxxxxxxxxxx>     
             org        
                     
                     
   cc 
                    
                     
                     
          
                    
                     
                     
  Subject 
             2008-03-10 15:27    
     [platform-core-dev] RE: [cdt-dev]   
                    
                  First gotcha
with add/exclude       
                    
                  children
of FFS                    
             Please respond to    
                     
                  
             "Eclipse Platform  
                     
                    
              Core component  
                     
                     
             developers list."  
                     
                    
             <platform-core-de  
                     
                    
              v@xxxxxxxxxxx>  
                     
                     
                    
                     
                     
          
                    
                     
                     
          




Copying the platform-core-dev folks too. Is there someone from the Platform
who could attend a meeting at EclipseCon about flexible file systems. John
A, will you be there? John is Mr. EFS and we can use his guidance.

Thanks,
Doug.

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Ken Ryall
Sent: Saturday, March 08, 2008 5:08 PM
To: CDT General developers list.
Subject: Re: [cdt-dev] First gotcha with add/exclude children of FFS

Doug,

Can we get together at EclipseCon to discuss this issue specifically? Do
you know the right platform people to invite to the meeting? We really
need to piece together a plan.

I'm sure you have enough to do so if you can tell me who to recruit I can
help organize the meeting.

Thanks - Ken

From: "ext Schaefer, Doug" <Doug.Schaefer@xxxxxxxxxxxxx>
Reply-To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Date: Wed, 20 Feb 2008 18:44:57 -0800
To: "CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Subject: RE: [cdt-dev] First gotcha with add/exclude children of FFS

So, you know. The more I think about what you guys are saying, I'm realizing
that the EFS solution probably is the right one. The objective should be
to turn the IResource tree into a logical project view and to remove all
notions that it represents physical file layout. That unfortunately starts
with the .project and .cproject files, but I think there are tricks we
can do there. The .settings may be harder but let me sleep on that.

At any rate, this has piqued my interest again and I'll work on reviving
it and see where it goes. I'll try to get a prototype working by EclipseCon
so we can talk about it more concretely.

Cheers,
Doug

From: cdt-dev-bounces@xxxxxxxxxxx [mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Brunauer, Walter
Sent: Monday, February 18, 2008 8:53 AM
To: CDT General developers list.
Subject: RE: [cdt-dev] First gotcha with add/exclude children of FFS

Hi Warren,

well, the confusion my origin from the different meanings of what project
setup is: for me, project setup is not equal to build setup. I.e., on our
projects, the build setup is an independent step from the project setup.
We intentionally separated this to overcome all kind of issues you obviously
experienced as well. And this is how we see it:

1. Create a project at the desired location (everything beneath this root
location is part of the project, but it can be an empty project just as
well with linked resources added to it later). By default, the build setup
is identical to the project content (there is one build target, linking/archiving
everything together).

2. If (a) specific build setup(s) are needed, it is possible to specify
as many build targets with arbitrary contents as desired. This approach
separates the physical file system layout from logical build layouts, and
it even works beyond project boundaries. IOW, no matter from where source
files are pulled in (the same projects, nested projects, outer projects,
sibling projects), one is able to specify any build setup exactly are needed,
as long as all source files are known to Eclipse (as resources).

HTH,

Walter



      From: cdt-dev-bounces@xxxxxxxxxxx  [
      mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of
      Warren.Paul@xxxxxxxxx
      Sent: Freitag, 15. Februar 2008  14:51
      To: cdt-dev@xxxxxxxxxxx
      Subject: RE: [cdt-dev] First  gotcha with add/exclude
children of FFS



      Hi Walter,



      I forgot all about the absolute paths issue with linked
 resources.
      I'll update the wiki.



      I'm a bit confused about your comment about this not
 being a project
      setup issue.  We have our own builder as well,
and it  will happily
      build whatever the build description says, whether
those files  are
      under the project root or not.  We even have
our own project explorer
      view which shows a logical representation of the project
rather than
      the  physical file system layout.  But we
still run into a lot of
      issues when  files are not under the project
root - that is, when you
      can't get an IFile  for them.



      We have a wide range of user types from small application
 developers
      to large system developers.  In many cases, a
users code base
      consists of hundreds of directories with thousands
of source files.
      In  such a source base, there are many hundreds
of build artifacts
      and almost as  many "logical projects".
 It is a huge problem for
      these users to be able  to create projects currently.
 They will
      typically have a few projects  going at a time,
but many times the
      natural project root for all of them will  be
the same.  We've found
      ways to work around some of the other issues,  but
not this one.  It
      sounds like perhaps you guys have found a  way.
 Could you elaborate
      on how you setup your  projects?



      Thanks,
      Warren



      From: cdt-dev-bounces@xxxxxxxxxxx  [
      mailto:cdt-dev-bounces@xxxxxxxxxxx] On Behalf Of ext
Brunauer,
      Walter
      Sent: Friday, February 15, 2008 1:56 AM
      To: CDT  General developers list.
      Subject: RE: [cdt-dev] First gotcha with  add/exclude
children of FFS



      Hi Warren,



      FWIW, you did not mention anything about linked  resources
and
      absolute paths these persist in the .project file
by default.  Again
      a big issue around linked resources in combination
with sharing
      project  within a team (even without team support),
and one more
      reason why they appear  to be so cumbersome to
handle. To me it seems
      many times one has  to unsell linked resources
to users: Whereas
      linked resources  are (kind of!) nice for evaluation
purposes
      (because, yes, in this case  you might not want
to pollute your
      sources), as soon as you start serious  development,
you run into all
      kind of troubles. The hurdle to get everything  right
from the
      beginning is overwhelming for novices (e.g. its not
possible to
      change a linked resource to use a variable later).
Sorry, I don't
      know how to  add this to the Wiki page...



      Having said that, the scenario you describe is  really
about having
      the flexibility around build and indexer setup, not
around  project
      setup, IMO.



      It's rather classic: users have common  code
they want to reuse in
      multiple applications - so they create one or  a
set of libraries out
      of it, within one or a set of projects. Of course,
 indexing should
      be able to handle only code going into these libraries,
and
      optionally ignore the rest. Then, they create their
application
      projects, which  use the binary artifacts of
the library project(s).
      Now it would be great if  they would have automatic
support for
      application linkage specification, i.e.  some
nice wizard or UI
      allowing to select the library binaries of other  projects
to be
      linked in, without the need to specify it manually
in the  linker
      options. And probably also desired: during application
code
      development, the public API's of all used library
projects should be
      the only  thing they see WRT code completion,
etc. I guess, some UI
      would be needed for  this as well.



      And now think of all developers in the world.  Wouldn't
it be great
      to give as many of them the freedom to choose how
to  achieve this?
      Either everything in one project, or one project per
build  artifact,
      or one project per module/application/product, or
with nested
      projects... its possible. Our commercial IDE based
on CDT supports
      all this,  and we did not have to provide some
EFS or work with
      linked resources.  Well, we had to override the
build system, and
      this is IMO the place to solve  this in CDT as
well.




      Again, I don't see anything specific to project  setup.
The issue
      around having the source tree polluted with project
files - I  don't
      think this is the big thing. I would not leave the
Eclipse path in
      this  area at all and allow to separate the project
file from the
      project location.  Its a very general paradigm
of Eclipse, and I am
      pretty sure doing everything  differently will
generate lots and lots
      of problems in all kind of areas  (probably much
more than you
      already identified), unless you make it a new  Eclipse
way
      (add/change this in the platform, not in CDT, that
 is).



      Just my 2 or 3 cents again,



      Walter




            From: cdt-dev-bounces@xxxxxxxxxxx
 [
            mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of
            Warren.Paul@xxxxxxxxx
            Sent: Donnerstag, 14. Februar
2008  23:35
            To: cdt-dev@xxxxxxxxxxx
            Subject: RE: [cdt-dev]  First
gotcha with add/exclude children
            of FFS



            I've updated the Wiki page
            http://wiki.eclipse.org/CDT:Flexible_Project_Structure
with
            some more thoughts on the issue.
 It would be great to get
            feedback  from other CDT
users - both those shipping C/C++
            IDE's and end users.   You'll
see that I'm not convinced that
            the linked resources route is
a  viable option.  Maybe we can
            get the platform team involved
in the  discussion to help find
            the best route forward.



            Thanks,
            Warren





            From: cdt-dev-bounces@xxxxxxxxxxx
 [
            mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of ext Schaefer,
            Doug
            Sent: Friday, January 25, 2008
11:15 AM
            To: CDT  General developers
list.
            Subject: RE: [cdt-dev] First
gotcha with  add/exclude children
            of FFS



            I guess what my investigation
has shown me that the EFS
            solution and linked resources
are pretty much identical. I
            really  noticed this when
trying to figure out how to persist
            the adds and found  myself
wishing I could add that to the
            .project file just like linked
 resource are. And they are....



            I think all the issues that we
have with linked resources would
            be equally as bad with the EFS
solution,  possibly worse
            because the EFS adds are hidden.
The CVS one is a great
            example. I really doubt CVS would
work with the EFS solution
            either. And I  don't want
us to think EFS would be better since
            it's not in the platform  where
we'll have a battle getting
            changes. Any platform changes
required to  make linked resource
            work correctly would also need
to be done for  EFS.



            So my hope is to save the effort
at implementing  the
            add/remove functionality since
I believe that's already there
            with  linked/hidden resources.
We can then focus on making
            linked resources  work where
we need them and improving the
            workflows. But this  really
needs to start now.



            So, Warren, you've somewhat started
a list of  workflows that
            we'd like to support with this
solution. This is a great  place
            to start. I've created a Wiki
page to start capturing these.
            Please  feel free to add
more information, especially to the
            workflow section. When  we
have that we may get a better idea
            of which of the two solutions
will  work best.



               http://wiki.eclipse.org/CDT:Flexible_Project_Structure



            Thanks,

            Doug





            From: cdt-dev-bounces@xxxxxxxxxxx
 [
            mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of
            Warren.Paul@xxxxxxxxx
            Sent: Friday, January 25, 2008
11:22  AM
            To: cdt-dev@xxxxxxxxxxx
            Subject: RE: [cdt-dev] First
 gotcha with add/exclude children
            of FFS



            We've been working on Eclipse/CDT
based products for  about
            three years now.  I'm sorry
to say that the project model is
            still not satisfactory for our
purposes.  We've tried many
            angles, but  are still stuck
with some pretty serious
            limitations.  I've volunteered
 to investigate the EFS route to
            see if it will help at all.  Based
on  this thread I'm assuming
            it won't.



            Let me give you a brief overview
of how our users work,  and
            then discuss the problem we've
run into.  I don't think any of
            this  is specific to our
users BTW.



            Most of our users have existing
code bases.  They  simply want
            to "import" it into
the IDE.  Others will create new  projects
            from our templates.  The
new projects are created in the
            workspace.  Imported projects
could be anywhere in the file
            system.  Often times they
will import several projects from the
            same  source tree.  This
is where our biggest problem is.
            Let's say the  source base
looks like this:



            C:\MyProjects\Project1\...


            C:\MyProjects\Project2\...


            C:\MyProjects\Common\...



            Because both projects  share
code in the Common directory, the
            logical root project directory
for  both Project1 and Project 2
            is C:\MyProjects\.  But
in Eclipse you  can't have two projects
            with the same root.  This
is where the .project  and .cproject
            files are created.  So currently
our users would import
            Project1 with the natural root
(C:\MyProjects\), but Project2
            has to be  rooted at C:\MyProjects\Project2\.
 This means that
            any source/headers  from
the common directory are not under
            Project2.  This means those
 files are not in the project
            explorer for that project, are
not indexed,  etc..  We logged
            this against the platform -
            https://bugs.eclipse.org/bugs/show_bug.cgi?id=78438.
            Basically if you put the .project
file anywhere, but have a
            project root  attribute,
this would cease to be a  problem.



            Our first product actually always
created the  .project in the
            workspace, and for imported projects,
would create links to
            files and folders.  We ran
into so many issues with this that
            we had to  change the model.
 I don't recall all of the issues,
            but here's a list  of some:



            - Version control simply didn't
work at  all



            - You can't make file system
changes with  links.  For example,
            if you want to rename a file
or folder, or move a  file around,
            you can't do this with linked
resources.  It only changes  the
            link itself, not the underlying
 resource.



            - Creating new resources in a
project with links is  confusing
            at best.  Let's say you
have a project with a linked folder
            and file at the root.  If
you create a new file or folder at
            the root,  it is created
in the workspace, not where the other
            folder/file are in the  file
system.  But if you create a new
            file under the linked  folder,
it gets created where you'd
            expect.



            - The location of the .project/.cproject
files are
            problematic.  Some users
will want to keep these in version
            control,  while others won't.
 Those that do want them created
            in the source  tree, but
those that don't want them elsewhere,
            like the workspace.  I  forget
now why this was a problem with
            linked resources, but there was
 something weird going on
            there.



            I suppose it's worth noting that
the last time we  really
            looked at this was in Eclipse
3.2, so some of this may have
            been  fixed by now.  But
I doubt it.  In general linked
            resources are  second class
citizens.  Some IResource API's
            just don't work for linked  resources.
 Just search for
            references to IResource#isLinked
for  "special handling".  I
            suspect that we'll run into similar
issues with  EFS though -
            see getLocation vs  getLocationURI.



            We also have the same issue that
Doug is trying to  address
            (hiding some branches in a source
tree).  This is much less of
            an issue for us though.  You
can already reduce the scope of
            the  indexer and the build.
 The only real issue for us is for
            a very large  source tree,
you're going to get IResource's for
            everything, which slows  things
down quite a bit.  There is
            actually somewhat of a problem
in  reducing the indexer scope -
            see https://bugs.eclipse.org/bugs/show_bug.cgi?id=178159.



            The hidden attribute addition
sounds promising for  hiding
            resources under the project root,
but doesn't really do
            anything to  add flexibility
to the contents of a project.  EFS
            sounds like it would  though.
 What I mean by that is, having
            resources under a project that
 are real resources, not linked,
            but that don't live under the
project root  in the file system.
            I've just started looking into
EFS, so maybe it's  a bit of
            wishful thinking at this point,
but I'm hoping we could create
            a  project anywhere, and
when we create it we pass the URI
            location from our  own EFS.
 Then when asked for the children,
            we could return URI's for  files
from anywhere in the file
            system, or on other machines
even.   This would seem to hold
            the potential for working around
the issues listed  above.
            We'd basically have an EFS map
from what we want under a
            project to the actual file  system.



            So hopefully some of the experts
can chime in  here.  Is my
            hope for EFS unrealistic?  Is
there a different  approach we
            should look at?



            Thanks,
            Warren





            From: cdt-dev-bounces@xxxxxxxxxxx
 [
            mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of ext Brunauer,
            Walter
            Sent: Friday, January 25, 2008
1:47 AM
            To: CDT  General developers
list.
            Subject: RE: [cdt-dev] First
gotcha with  add/exclude children
            of FFS



            Hi,



            after reading this rather long
thread, I'll  decided to throw
            in my personal opinion.



            I consider this approach to work
against one of  the most
            general Eclipse platform paradigms,
where a project is defined
            to be a root directory and everything
in it. IMO, the more
            workarounds are  introduced
against this paradigm, the more
            problems will be faced, and the
 more incompatibilities (or at
            least unawarenesses)  created.



            Isn't the whole problem you try
to solve here  rather about
            what files should go into the
build (and probably into the
            indexer) than what files are
part of a project? I understand
            that CDT has no  separation
of what a project and what the
            build input is (well, IIRC one
can  exclude specific files from
            the build, but in general, the
project content  defines the
            build input, right?).



            In our commercial IDE, we separated
this. This not  only
            introduced much more powerful
build setup capabilities in
            general, but  especially
enabled users to setup build artifacts
            with arbitrary  contents
(think of sources being compilable
            with different compiler flags
 for different build artifacts,
            build input exclusion patterns,
build input  from all over the
            workspace, multiple build artifacts
within the same  project,
            reusable build artifacts accross
project boundaries, etc.,
            etc.,  etc.). BTW, we call
this build system flexible managed
            build - because  that's
what it is:-)



            Of course, one can setup CDT
projects as of today  to exactly
            contain what is desired (with
the help of linked resources).
            However, I find linked resources
to be cumbersome and error
            prone, though  many of our
customers start out with them during
            evaluation as well, mostly  because
they are looking for a way
            to achieve what they did in the
past with  other non-Eclipse
            based IDEs, but sooner or later
I know of lots of  them
            realizing its much easier to
use the features of our flexible
            build  system instead, especially
if projects need to be shared
            in a team.  And now think
of virtual file systems, the
            potential complexity of  these,
hidden assumptions,
            restrictions, etc. Sounds worse
than linked  resources to me.



            I guess, the point I am trying
to make is:  whatever you decide
            to do, make it understandable
 and transparent (and of course
            as simple as possible to use)
for  the user.



            As said, just my 2 cents,



            Walter




                  From: cdt-dev-bounces@xxxxxxxxxxx
 [
                  mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of
                  Schaefer,
 Doug
                  Sent: Donnerstag,
24. Jänner 2008 23:17
                  To: CDT
 General developers list.
                  Subject:
RE: [cdt-dev] First gotcha  with add/exclude
                  children
of FFS



                  Jogging
through the code, it really looks like the
                  HIDDEN feature
is what I was looking for. What I haven't
                  found yet
is UI  to make a resource hidden or a navigator
                  filter to
show hidden resources  (in case you want to
                  bring them
back). Is this planned?



                  Assuming
we have the core features available to link  in
                  and hide
resource, I think we still have workflow issues
                  that need
to  be addressed. I like Ken's idea of a file
                  that controls
the  linking/hiding. We could have an
                  import/export
mechanism for setting up  projects based on
                  this file.
A nice wizard for creating the file would
                  also be
good, similar to the way the way the export file
                  system wizard
 works.



                  Given this,
we may be further along than we thought
                  (BTW, the
hidden stuff seems to have been added in 3.4
                  M4).



                  Cheers,

                  Doug



                  From: cdt-dev-bounces@xxxxxxxxxxx
 [
                  mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of
                  Schaefer,
 Doug
                  Sent: Thursday,
January 24, 2008 2:51 PM
                  To: CDT
 General developers list.
                  Subject:
RE: [cdt-dev] First gotcha  with add/exclude
                  children
of FFS



               
  Thanks Michael/Szymon,



                  Is there
a bug describing the isHidden  feature?


                  Doug



                  From: cdt-dev-bounces@xxxxxxxxxxx
 [
                  mailto:cdt-dev-bounces@xxxxxxxxxxx]
On Behalf Of Michael
                  Valenta
                  Sent: Thursday,
January 24, 2008 11:37 AM
                  To:  cdt-dev@xxxxxxxxxxx
                  Subject:
RE: [cdt-dev] First gotcha with  add/exclude
                  children
of FFS


                  Doug et
al,

                  Szymon is
really the person you want to bug  on this but
                  I'll throw
in my 2 cents ;-)  First, I have to say that a
                  solution
at the IResource level (e.g. using linked
                  resources
and the new  hidden folder support) is
                  infinitely
better from a repository provider  perspective
                  than an
EFS based solution.  You may not get all the Team
                  support
you want at the IResource level but a solution at
                  the EFS
level  would certainly break the existing CVS
                  client since
the CVS client isn't  EFS aware to any great
                  extent.
For instance, if you tried to hide a folder
                  using EFS,
the CVS client would probably try and recreate
                  it the next
time  you performed a Team>Update. It is also
                  important
to note that the  Platform does not provide all
                  the hooks
required by repository providers  and I know of
                  at least
one provider that has resorted to using it's own
                  EFS implementation
under projects that are mapped to that
                  provider
to get  the capabilities it requires. I think it
                  is important
that tooling in  Eclipse stick to using the
                  IResource
layer as the layer they operate on  and let the
                  repository
provider (or any other tooling whose
                  responsibility
 it is to manage the available files)
                  control
the underlying file system.  If there are
                  shortcomings
or enhancements required then you should
                  push to
 get them in at the IResource level.

                  As for the
current state of  Team support for linked
                  resources,
I think the best approach is to  enumerate
                  some specific
scenarios of how you see linked resources
                  and  exclusions
working with descriptions of what you
                  need to
do today to get  Team support and what you would
                  like to
see. It is also important to know  if you expect
                  all the
links to come from the same repository (or at
                  least  repository
type) or whether a project could
                  contain
content from different  repository types
                  (obviously
the later would be more difficult to
                  accommodate
than the former).

                  Hope this
 helps,
                  Michael


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev
_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev
_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev