Re[2]: [equinox-dev] Good support for PDE in build systems anyone?

[Peter Kriens <Peter.Kriens@xxxxxxxxx>]

JM> Metanote:  the whole 1:1 project:bundle mapping issue revolves
JM> around our desire to present at compile time, a classpath that is
JM> as close as possible to the classpath that will be used at
JM> runtime.  Since we build a project at a time and each project has
JM> one classpath, you get the current behaviour.  
I have complained many times about this philosophy, the cost
of having no-build phase is much higher than the second it potentially
gains. It causes redundancy, which imho costs you more in the long
run.

The whole idea of the OSGi has been to create components that work in
a large range of environments. You do not know the classpath of your
target by definition. That is why package-import is so important
because it reduces the requirements on your environment. It is a
fallacy to think you can accurately mimic your run-time during
development time when the whole goal is to allow many run-times.

For example, the OSGi allows import of version ranges. The PDE could
support a classpath that has alternatives for JARs so you
tests that compiles your code against the permutation of alternatives
and from the results generates the import ranges.

There are lots of scenarios like this that can save
significant amount of work but they all require something to happen
between developing and running.

JM> To date the value of having an accurate classpath at build time
JM> has outweighed any issues wrt the mapping.
Maybe for you ... after converting many projects to PDE they are now all
back to my home grown build system because it was too painful :-(

Kind regards,

     Peter Kriens







JM>
JM> Peter Neubauer wrote on 06/04/2006 03:47:51 PM:
JM>  
 >> in particular I find the use of Maven plugins and other tools not  
 >> sufficient when it comes to developing OSGi bundles in the Eclipse  
 >> PDE. The problem is, compile time everything is fine having  
 >> dynamically mounted jars via virtual classpath containers a la  
 >> MevenIDE, and the old Magic plugin I wrote. But even so, PDE requires  
 >> a valid Manifest to be present under /meta-inf of the current OSGi  
 >> bundle project in the workspace, and the PDE dependencies virtual  
 >> classpath variable mounts the deps mentioned in the Manifest.
 >> 
>> However, when you try to run/debug your plugin, the dependencies  
 >> stated in the Classpath entry in the manifest have to be there under  
 >> the bundle root folder.
JM>   
JM> It is conceivable that the manifest be somewhere else.  Not sure
JM> what exactly the issue is here.  That is, if you have a manifest
JM> in your project in your workspace, is it an aesthetic thing that
JM> would cause you to put is somewhere other than in META-INF?  or is
JM> there a technical issue?  As you note, this is not just PDE being
JM> anal.  When you turn around and want to run your bundles, we like
JM> to avoid any sort of build/deploy/...  so if the manifest is going
JM> to be somewhere else, the framework you are running still has to find it/be told about it.
JM>  
JM> Metanote:  the whole 1:1 project:bundle mapping issue revolves
JM> around our desire to present at compile time, a classpath that is
JM> as close as possible to the classpath that will be used at
JM> runtime.  Since we build a project at a time and each project has
JM> one classpath, you get the current behaviour.  To date the value
JM> of having an accurate classpath at build time has outweighed any issues wrt the mapping.
JM>  
>> Basically, the problem is that the PDE is taking the Manifest and  
 >> OSGi as master, where maven and others are generating everything from  
 >> a POM/Ivy file.  
JM>  
JM> Yes, this is a characteristic that Peter Kriens has been pointing
JM> out.  Certainly there are cases where you need metadata at
JM> development time that is not needed at runtime.  Having an
JM> additional file to hold this information is a good idea.  PDE uses
JM> build.properties.  Somewhat crude but functional.   
JM>  
JM> The point about wanting to run without build/deploy is relevant
JM> here too.  PDE has several mechanisms that work to keep the
JM> manifest "runnable" at all times.  They also feed back
JM> error/warnings as you make the problems so there are fewer surprises.  
JM>  
JM> Increasingly people are using the PDE graphical editors (as
JM> opposed to just the source page) so in a sense the underlying
JM> format of the data and what file it is in is decreasingly
JM> important to PDE.  (of course Wassim and the folks who actually
JM> write that code may have a different opinion :-)  However, it is
JM> not clear that coming up with N different syntaxes to represent
JM> what is already so well set out in the spec makes sense.  The
JM> maven-osgi plugin has its own way of spec'ing the dependencies.
JM>  Not sure about Ivy.  Why not just use the manifest? 
JM>  
JM> There was some very initial discussion with some Maven folks
JM> about the possibility of having Maven read manifests directly and
JM> use them to populate the internal model.  Nothing has happened there AFAIK.
JM>  
>> In Ivy, we have solved that somewhat by
 >> - reading most of the attributes from the Manifest.mf as master  
 >> instead of the ivy.xml
 >> - download the different deps locally before compiling and using them  
 >> for the classpath from within the PDE. Generated by Ivy but still in  
 >> harmony with the PDE we have a working build system with Ivy that  
 >> works good with the PDE at https://scm.ops4j.org/repos/ops4j/ 
 >> laboratory/users/andreas/builder/
 >> - package the bundle with a similar structure as the development layout.
JM>   
JM> Is this working for you?  What are the gotchas? 
JM>  
JM> BTW, what is hte license on Ivy? 
JM>  
>> I'm not sure how to solve this discrepancy between the two mindsets
 >> - runtime, OSGi rules and the PDE tries to enforce valid development  
 >> time editable artifacts
 >> - metadata rules and all runtime data is generated from POM/Ivy  
 >> metadata, which is the build system approach.
 >> 
 >> I see two major points where the PDE has to be adapted to honor the  
 >> build system or the other way round:
 >> 
 >> 1) Manifest file
 >> - one could think of a Maven builder generating new manifest.mf on  
 >> the fly as you edit the relevant POM/other metadata, like source code  
 >> triggering changed import statements 
JM>  
JM> Sure.  Note that PDE does not have to do anything here.  It will
JM> pick up any changes made to the manifest by such a builder. 

 >> - flexible location of the Manifest in the bundle or at least  
 >> allowance in the PDE to specify a manifest by URL (to be able to  
 >> reference it as a custom protocol "artifact:mf:my/project/ 
 >> manifest#1.0.2") pointing to a maven resolved repository or generated  
 >> location.
JM>   
JM> No doubt the implementation of PDE is expecting the manifest ot
JM> be in the /META-INF dir but aside from that, conceptually the
JM> manifest might be somewhere else at development time.  However, at
JM> runtime someone is going to have to either put it in the standard
JM> spot or tell the framework where it is. 
JM>  
>> 2) Dependencies
 >> - not sure that is possible but it would be great to be able to  
 >> reference everything in the PDE as URLs, opening for mounting e.g.  
 >> the runtime jars from the local maven/ivy repo into the right place  
 >> for the PDE. Somewhat possible with Eclipse workbench links I guess  
 >> but they seem to be tied into absolute file paths and protocols,  
 >> making them too static for this.
JM>   
JM> PDE works on the notion of a "target platform".  This is a set of
JM> bundles that are in the filesystem and not in your workspace.
JM>  Think of it like the Maven local repo.  In 3.2 the target
JM> platform can source bundles from several locations and the
JM> expected structure of these locations has been relaxed somewhat.
JM>  This is not enough as the Maven, for exmaple, structure is
JM> potentially deep and PDE does not search.  One also has to
JM> consider whether or not you want the whole local repo exposed as
JM> part of the target.  Typically the target is used to scope down
JM> what you are expecting to be available...  In any event, there has
JM> been some work in the direction.  See  
JM>         https://bugs.eclipse.org/bugs/show_bug.cgi?id=124542 
JM>  
>> - best would be to be able to even in the manifest.mf bundle  
 >> classpath to use URLs for the private jars etc. in order to be able  
 >> to use the same approach
 >> - this would enable things like Hansa/Transit that is Niclas favorite  
 >> pet ;) http://wiki.ops4j.org/dokuwiki/doku.php?id=hansa:hansa
JM>   
JM> by "private jars" do you mean the ones on the classpath of the bundle?
JM>  
JM> Some of the above issues may actually turn out to be JDT things
JM> in as much as all PDE is really doing is populating a JDT
JM> classpath.  I don't think that they support URLs in any form on
JM> the classpath.  No idea what the technical issues would be there. 
JM>  
>> Not sure how much of this sounds understandable and how much there is  
 >> support for it in the OSGi spec and Eclipse PDE possibilities, just  
 >> thinking aloud.
 >> 
 >> WDYT? does anyone out there have a nice working PDE system with Maven  
 >> working for OSGi bundle development and a description of the process?
JM>   
JM> Certainly allowing more people to work in the way they are
JM> accustomed to is a good thing.   
JM>  
JM> Jeff 

-- 
Peter Kriens                              Tel +33467542167
9C, Avenue St. Drézéry                    AOL,Yahoo: pkriens
34160 Beaulieu, France                    ICQ 255570717
Skype pkriens                             Fax +1 8153772599



-- 
Peter Kriens                              Tel +33870447986
9C, Avenue St. Drézéry                    AOL,Yahoo: pkriens
34160 Beaulieu, France                    ICQ 255570717
Skype pkriens                             Fax +1 8153772599