Bug 438526 - [P2] support GIT source location in target definition.
Summary: [P2] support GIT source location in target definition.
Status: REOPENED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.4   Edit
Hardware: PC Mac OS X
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: PDE-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: helpwanted
Depends on:
Blocks:
 
Reported: 2014-06-30 05:08 EDT by Christophe Bouhier CLA
Modified: 2015-09-23 07:02 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christophe Bouhier CLA 2014-06-30 05:08:44 EDT
In the spirit of Ruby On rails, Bundler -> git support:

gem 'forem', :github => 'radar/forem', :branch => "rails4"

Dependencies in Ruby On Rails are in a so called "Gemfile", can get them by using a github location.  

more reading: http://bundler.io/


An equivalent for Eclipse target, could be created for a .target file source location. 

why: well simply cut-out an intermediate step, which is a P2 repo, when we can get plugins/features from the source.
Comment 1 Paul Webster CLA 2014-07-02 10:10:58 EDT
To load source into your workspace you use a PSF file, not a target definition (which is, by definition, binary artifacts).

Are you asking for a way to fill in a target definition but have PDE load source for some bundles into your workspace?

PW
Comment 2 Christophe Bouhier CLA 2014-07-02 10:21:28 EDT
Paul, 

This is not about loading source in the workspace, but about a target source location being a git repo containing features/plugins. These would be loaded in the target. (In the same way the plugins in the workspace end up in the target). 


So what I mean is this, see the repository location.  


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?pde version="3.8"?><target name="test" sequenceNumber="0">
<locations>
<location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">
<unit id="" >
<repository location="https://github.com/dzonekl/netxstudio.git"/>
</location>
</locations>
</target>
Comment 3 Curtis Windatt CLA 2014-07-02 10:25:16 EDT
For the Eclipse SDK (and any other bundles that include the Eclipse Source Reference header), you can already import the git repo source for bundles in your target.

File > Import > Plug-ins and Fragments
Import from: Active target platform
Import As: Projects from a Repository

This will search your target platform for bundles with a source reference header.  That header specifies the git (or CVS or SVN) location to get the source for the project.
Comment 4 Curtis Windatt CLA 2014-07-02 10:27:10 EDT
(In reply to Christophe Bouhier from comment #2)
> Paul, 
> 
> This is not about loading source in the workspace, but about a target source
> location being a git repo containing features/plugins. These would be loaded
> in the target. (In the same way the plugins in the workspace end up in the
> target). 
> 
> 
> So what I mean is this, see the repository location.  
> 
> 
> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
> <?pde version="3.8"?><target name="test" sequenceNumber="0">
> <locations>
> <location includeAllPlatforms="false" includeConfigurePhase="true"
> includeMode="planner" includeSource="true" type="InstallableUnit">
> <unit id="" >
> <repository location="https://github.com/dzonekl/netxstudio.git"/>
> </location>
> </locations>
> </target>

p2 does not recognize a git repository as a valid repository.

While a new location type could be created for Git, this would not be effective as the target platform requires the built binary bundles, not the source.

Closing as INVALID.
Comment 5 Christophe Bouhier CLA 2014-07-07 08:29:11 EDT
(In reply to Curtis Windatt from comment #4)
> (In reply to Christophe Bouhier from comment #2)
> > Paul, 
> > 
> > This is not about loading source in the workspace, but about a target source
> > location being a git repo containing features/plugins. These would be loaded
> > in the target. (In the same way the plugins in the workspace end up in the
> > target). 
> > 
> > 
> > So what I mean is this, see the repository location.  
> > 
> > 
> > <?xml version="1.0" encoding="UTF-8" standalone="no"?>
> > <?pde version="3.8"?><target name="test" sequenceNumber="0">
> > <locations>
> > <location includeAllPlatforms="false" includeConfigurePhase="true"
> > includeMode="planner" includeSource="true" type="InstallableUnit">
> > <unit id="" >
> > <repository location="https://github.com/dzonekl/netxstudio.git"/>
> > </location>
> > </locations>
> > </target>
> 
> p2 does not recognize a git repository as a valid repository.
Yes, the point of this bug is to support this.  

> 
> While a new location type could be created for Git, this would not be
> effective as the target platform requires the built binary bundles, not the
> source.
How true is this statement, when the plugins in my own workspace (Which are not binary) are mixed into the Target Platform. 
> 
> Closing as INVALID.
It seems to me the bug is closed very quickly.  I started this bug with a comparison from another framework where this is common practise and very appreciated I can assure. What would you need to be convinced? More uses cases, votes etc..?
Comment 6 Paul Webster CLA 2014-07-07 08:59:19 EDT
(In reply to Christophe Bouhier from comment #5)
> > 
> > While a new location type could be created for Git, this would not be
> > effective as the target platform requires the built binary bundles, not the
> > source.
> How true is this statement, when the plugins in my own workspace (Which are
> not binary) are mixed into the Target Platform. 

But they're in your workspace.  A mechanism to load bundles from the target platform into your workspace exists already, under File>Import...>Plug-in Development>Plug-ins and Fragments.  You can import them as source, and import them from their repository (usually git).

PW
Comment 7 Christophe Bouhier CLA 2014-07-07 09:49:36 EDT
(In reply to Paul Webster from comment #6)
> (In reply to Christophe Bouhier from comment #5)
> > > 
> > > While a new location type could be created for Git, this would not be
> > > effective as the target platform requires the built binary bundles, not the
> > > source.
> > How true is this statement, when the plugins in my own workspace (Which are
> > not binary) are mixed into the Target Platform. 
> 
> But they're in your workspace.  
so, a git source locations could resolve in the workspace as well. 
I use target files with tycho. I can easily imagine the IU's from a git location end up in the workspace as well. This would especially make sense with a headless build using tycho and hudson/jenkins.

Perhaps in the Ruby world it works, because there are no compiled/linked binaries...

A mechanism to load bundles from the target
> platform into your workspace exists already, under File>Import...>Plug-in
> Development>Plug-ins and Fragments.  You can import them as source, and
> import them from their repository (usually git).
> 



> PW
Comment 8 Paul Webster CLA 2014-07-07 10:13:10 EDT
(In reply to Christophe Bouhier from comment #7)
> so, a git source locations could resolve in the workspace as well. 

As an enhancement, it's possible that target files could be enhanced with a "and load from the EclipseSource header into workspace" for loading a target platform into the IDE with PDE.  Although that seems like something you'd want as an import switch/check-box to allow the user to control (I wouldn't want the source for my target platform, except in rare circumstances).


> I use target files with tycho. I can easily imagine the IU's from a git
> location end up in the workspace as well. This would especially make sense
> with a headless build using tycho and hudson/jenkins.

Tycho is a different component, and not part of PDE.  It doesn't have a workspace, so there's no place to grab the source. I also suspect loading the target platform happens at a maven build step/phase that might make it difficult to suddenly materialize source in the current maven reactor (where it would have to be in order to be compiled).

If you want a target platform to fetch and compile source in tycho, you should open a bug against them though, and see what they have to say.

PW
Comment 9 Christophe Bouhier CLA 2014-07-08 08:48:14 EDT
Hi Guys, 

What I really would like to have addressed, is outdated P2's or in other words compile against the latest in a git repo. We now have a build cycle, where every library/project has to produce an integration type of p2, which is by nature always behind the source code and even worse, it's an additional artifact, which has to be produced, published,  maintained and found. I spend a great deal of time understanding which P2 I need, while many sources is in git.eclipse.org with a unified URL naming. P2's come and go. Recently, I nailed my target on plugins (I checked some specific plugins versions) in the xtext p2 repo. Luna was released, and these plugins where not offered anymore by the P2. Eclipse complained about it. Why do I need to be confronted with this? Just hook me into the source, with tags for the version I need.  

The materialization of a project, is as a step which can easily be taken by the final producer of a SW product (The guy at the end of the production chain).  

Now: 

lib1 -> lib1.p2 -> lib2 -> lib2.p2 -> app1 
        (find,wait,update)             (find,wait,update). 

Anticipated: 

lib1 -> lib2 -> app1. 
        (lib2 target includes src lib1 etc..) 


The EclipseSource header, could achieve this, but non optimal.  (The reason it was introduced, addresses a similar need, which is stay close to the source!). On the Eclipse IDE, it could work like this: 

" Hey I have a binary, but there is a newer source(judging from last commit?), do you want to pull it in the workspace, to be more on the edge? We will update the target state to compile your own projects against it"  (Can we hide this part of the workspace please? we don't want to change the code, just use it!). 


On the tycho part, same story really. It ends up in a workspace which the Maven reactor will compile against. I will ask the google people of tycho to give their opinion on this. 


A more optimal user story would be to skip p2 location altogether... and do whatever is necessary in the IDE to make it work with Git locations and be non-intrusive. (So hidden workspace with source projects, support git locations in target files, ).. After thinking this through it makes more and more sense. P2's are instable particles. Git repos are solid has granite. 


Cheers Christophe
Comment 10 Curtis Windatt CLA 2014-07-08 11:05:30 EDT
So you want a target definition that can specify a list of projects from one or more git repositories.  This would automate the existing workflow of importing the projects into the workspace and doing a pull on each git repo.

I think it is possible to do this, by creating a new target location type.  This would have to be contributed work that PDE would review.

I see several potential roadblocks:
- Target platforms do not update automatically.  You would have to hook into the update button functionality to do a git pull request.
- PDE has no dependencies on eGit/jGit as it isn't part of the SDK. This would likely mean that this work has to be done in a separate plug-in not in the SDK.
- The layout of a project in git can vary significantly from the built binary bundle. It is quite possible that the project in git will not be recognized as a plug-in by PDE.  Most projects will work as long as PDE can find the manifest, but a git project can contain any layout.