[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [equinox-dev] Good support for PDE in build systems anyone?


I think we lost the OPS4J list in there somewhere.  Not sure that my posts were getting through anyway...  

Peter Neubauer wrote on 06/05/2006 03:04:53 PM:

> Mmmh,
> On Jun 5, 2006, at 10:01 AM, Niclas Hedhman wrote:
>
> > Perhaps Peter is interested in resurrecting Silk, which is an  
> > experiment from
> > last year. Key points;
>

> But that was not the point of my post. I am really interested in how  
> we can make the PDE more build systems friendly, which IMHO would  
> even result in making the whole build process for Eclipse based  
> projects and Eclipse more standard and straightforward.

yes, we like this idea as well.  

> Jeff:
>  >PDE works on the notion of a "target platform".  This is a set of  
> bundles that are in the filesystem and not in your workspace.  Think  
> of it like the Maven local repo.  In 3.2 the target platform can  
> source bundles from several ?>locations and the expected structure of  
> these locations has been relaxed somewhat.  This is not enough as the  
> Maven, for exmaple, structure is potentially deep and PDE does not  
> search.  One also has to consider whether or >not you want the whole  
> local repo exposed as part of the target.
>
> Additionally, you would need your own target platform for every  
> single bundle project, which of course the build system can generate,  
> but it would require "scoped" target platforms, which essentially  
> would be to open up the target platform in exactly the same way as  
> the dynamic classpath entries via a Container plugin concept, very  
> interesting thought that! After all, why not mount the correutn taret  
> platform approach in the same way as the JRE jars vai a default  
> bundle container, and open that up later? That is, take the OSGi  
> bundle development where the Java development tooling already is.
>
> Jeff:
>  >by "private jars" do you mean the ones on the classpath of the bundle?
> Yes exactly. Having them exposed as URLs would open for dynamic  
> population behind it depending on runtime/development/testing  
> scenario without compromising the integrety of the manifest model.

I do see what you are driving at however. It does get quite complicated however since eventually the compiler is going to need something to compile against.  The only thing that compilers know about are dirs of classes and JARs of classes.  See next comment...

>  >Some of the above issues may actually turn out to be JDT things in  
> as much as all PDE is really doing is populating a JDT classpath.  I  
> don't think that they support URLs in any form on the classpath.  No  
> idea what the >technical issues would be there
>

> Mmmh, I think I asked that before when I tried to hook Hansa/Transit  
> into Eclipse and run the whole beast of a Maven-like central  
> repository by registering another URLHandler into OSGi and working  
> with new URL syntaxes in the config.ini etc etc. It seems that in the  
> Platform there are a lot of file dependencies all over the place. Not  
> sure where to start the process of removing them, or whether there  
> are any strong opinions on why this is so. Any ideas there?

yes, this is true wrt run time.  Interestingly we had the tar beaten out of us for adding the ability to use external: to indicate Bundle-Classpath entries that are outside the bundle.  This was in support of usecases where, for example, JDBC is installed by some other system and you want to present it as a bundle but can't move or modify it.  This has several problems wrt modularity, update, ... but none the less solves a real problem.  I'm not sure what the reaction would be if we allowed arbitrary URLs on the bundle class path...

Since you ask however, actually the whole storage mechanism of Equinox is quite flexible and open to others to extend/customize.  You might be interested to look at the org.eclipse.osgi.baseadaptor.hooks package (in the defaultAdaptor source dir in the org.eclipse.osgi project) and in the org.eclipse.core.runtime.internal.adaptor package (in the eclipseAdaptor source dir).  Basically the Equinox framework as an "adaptor" layer that allows others to replace the undersides of Equinox (how bundles are stored, how classloading happens, etc).

Having said that, I had assumed here you were talking more about development time.  At development time you have to pass a JAR to the compiler on the classpath.  This means that someone in there has to be interpreting/resolving these URLs and arriving at an actual JAR to use.  That JAR cannot have nested JARs (compilers don't like that) or funny dir structures (ditto).  This can start to get quite complicated.  Interesting but complicated.

> Jeff:
>  >This is possible because PDE has a complete view of the dependency  
> information.  Certainly Maven can (and should) be updated to do  
> something similar in the way it handles Import-Package.  Actually,  
> what do Maven/Ivy do > today for Import-Package.
>

> Agree here. This is one of the strong points for PDE. Since it is  
> based on a running OSGi instance, it is the only tool out there that  
> has OSGi compatible information at development time. And there is no  
> way to mimic the power of that in an environment outside OSGi. So, in  
> the build systems everything is based on more or less duplicating the  
> dependency information from the manifest one way or the other, and  
> hard wire compile and test time jars. Merging this was part of SILK ;)

A *very* important point here.  PDE does *NOT* use the same state etc as is used by the current running Equinox framework.  The state and resolver used are the same *implementation* but not the same instances.  In the not so distant past when it looked like the Maven folks might be interested we did a pass of separating out the resolver as a standalone piece.  It was always designed with this in mind but there were some dangling references etc.  We added generic constraints and ended up with a full power, standalone (i.e, no OSGi framework required) state representation with resolver in single JAR.  In the end the Maven folks did not bite so the work to reproducably create the separate JAR has been shelved.  In any event we know of several people (other than PDE) who are using the state/resolver to model other systems.  See the org.eclipse.osgi.service.resolver package in the org.eclipse.osgi bundle/project.

> Jeff:
>  >> In Ivy, we have solved that somewhat by
>  >> - reading most of the attributes from the Manifest.mf as master
>  >> instead of the ivy.xml
>  >> - download the different deps locally before compiling and using  
> them
>  >> for the classpath from within the PDE. Generated by Ivy but still in
>  >> harmony with the PDE we have a working build system with Ivy that
>  >> works good with the PDE at https://scm.ops4j.org/repos/ops4j/
>  >> laboratory/users/andreas/builder/
>  >> - package the bundle with a similar structure as the development  
> layout.
>
> Thanks a lot for all the discussion so far, it seems I'm not alone  
> with the problems of getting contineous integration, version handling  
> and good development tooling to work nicely together.

Hey, you started this so thank you.  It is a very interesting area and one that is not well/completely served by any current build technology.

Jeff