[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[equinox-dev] Maven and all that [was: Jetty and commons logging]

[redoing the subject to be more accurate as we are well and truly off the original topic...]

That doc talks about "releases" but does not define the term.  Do you know what the expectation is?  One of our builds can be 200MB or so and we end up running at least one and sometimes multiple a day in some (unhappy) cases.  Last I knew ibiblio was not keen on this sort of publishing.  Our weekly integration builds would be a candidate though they might prefer only the 6week milestones.  Either way the process is the same.  We run several such builds during the integration/milestone week and in the end declare one of them as "good" and promote it.  We do not apriori know which build is going to be "the one" as a mess o' testing is done after the build (automated and manual).  A publishing step such as discussed here would have to be after the fact and so cannot be built into the build itself.  

What is really needed is a tool that would take an Eclipse build (e.g., update site or download zip or ...) and fire it into the desired repo in the desired form.  This would accommodate the myriad of different build mechanisms/processes that exist today at Eclipse and allow anyone who wanted the content in another repo to do so without having to depend on the particular project build team/process.  With this in hand we could talk about getting particular build teams to call the tool when they promote a build.  Alternatively and more generally it would be interesting to have an RSS feed (forgive me Kim) for build promotions.  External parties can then start sucking content as soon as it is "released".  


Hal Hildebrand <hal.hildebrand@xxxxxxxxxx>
Sent by: equinox-dev-bounces@xxxxxxxxxxx

01/06/2008 09:08 PM

Please respond to
Equinox development mailing list <equinox-dev@xxxxxxxxxxx>

Equinox development mailing list <equinox-dev@xxxxxxxxxxx>
Re: [equinox-dev] Jetty and commons logging

I guess I'm a little confused...  There's a few sites which will do the mirroring for you, like ibiblio.org.  Ibiblio, and the myriad of mirrors for the site, host a plethora of maven artifacts for projects all over.  I'm not completely up on the mechanisms for publishing to these repositories - i.e. what are the protocols, contacts, etc - but considering that plenty of projects and code bases publish there means it can't be too hard to do.  Further, tools exist for Ant and Maven to make this publishing part of the build release process.  Or, at the very least, a target of the build which can be manually called when appropriate.

So, it seems like everyone could be happy.  Those actually build outside of Eclipse (strange, I know) could have access to Eclipse artifacts from a maven repository hosted by Ibiblio rather than having to provide your own space, and you Eclipse folks would have this folded into your build in a painless fashion.

This link has the Apache documentation on publishing to their site which is mirrored by Ibiblio: http://www.apache.org/dev/release-publishing.html
Although this wouldn't be what you'd be doing, I'm fairly certain the process would be similar.

Hope this helps all involved...

On Jan 6, 2008, at 5:29 PM, Jeff McAffer wrote:

Hey Alex,

Please note that I started my reply by saying "I don't disagree".  The point is that the team available to do this sort of work is small and the diskspace/mirror capacity to do it is similarly restricted.  I'm not happy about it either.  As with many of these things, if there are folks in the community who care deeply about this and want to do the work/supply the space, we'd be happy to work with them.

You are also correct in observing that in the p2 work we did indeed decide to have separately downloadable repository (metadata and artifact) indexing.  Further our general approach has been to allow for individual bundle download/install rather than whole feature management.  These moves have been a big win and should go a long way towards addressing the valid usecases you point out.

I completley agree that the update site structure is lacking in these areas but disagree with you that Maven addresses the issues.  The POM file certainly can represent the metadata in a bundle but its primary focus is to drive the build.  So, for example, if the build generates the dependency information into the output manifest.mf (e.g., using bnd or mangen style processing) then that dependency info is not going to be in the POM.  The standard maven repo does not have an index so it still does not help you find something unless you know exact ids and version numbers.  If you have that information then you can get it from a Maven repo or Update site equally well. (yes, I realize that there is a Maven repo indexer.  This is akin to the update site digest but with a finer grain)

I don't mean to slam Maven at all.  It is a fine tool and many people like it.  In the course of the p2 work we have learned some things from Maven and a careful eye will observe some Mavenesque characteristics in p2.  The standard Maven repo structure however does not meet our requirements (we spent quite a bit of time with various Maven folks going over the possibilities here).   As a design point we opted out of defining "one" repo format in favour of allowing people to add their own.  This opens the door for people with a Maven (or other) bent to add the support they need.  Similarly, our default repo structure is not that complicated so someone in the Maven world should be able to write a Maven repo class to manage p2's simple repos or even use p2 API to write them (thus mapping onto any p2 repo).

Anyway, we are way off topic here.  It suffices to say that the orginal poster's problems were caused by someone creating and publishing a faulty commons.logging bundle.  I doubt that was created by the eclipse team or if it was, it was a prerelease version (the Europa version has the necessary dependency version numbers).  In the ideal world it would actually be the Apache folks who produced these bundles and we would all just consume them.  But I'm ahead of myself...


"Alex Blewitt" <alex.blewitt@xxxxxxxxx>
Sent by:

01/06/2008 01:35 PM

Please respond to
Equinox development mailing list <

"Equinox development mailing list" <equinox-dev@xxxxxxxxxxx>
Re: [equinox-dev] Jetty and commons logging

On Jan 6, 2008 1:18 AM, Jeff McAffer <
Jeff_McAffer@xxxxxxxxxx> wrote:
> I won't disagree and would certainly welcome anyone who wanted to
> create/maintain such a repo.  It seems that someone has done this at
> maven.org.  The Maven user community should encourage that person to
> continue doing the work.  I don't really see eclipse.org or the projects
> hosting another copy of all their bundles laid out in the standard Maven dir
> structure.  Our mirrors would be upset at the doubling in size and we have
> to maintain the update site structure for the Eclipse user community anyway.
> As I understand Maven, it is at least in theory possible to define different
> kinds of repos so, for example, Eclipse update sites could be directly
> accessed from Maven.  No idea what is involved in that but it feels like the
> right kind of direction.

The problem with the Eclipse downloads is that there's actually no
real way of knowing where to look for something. Quite apart from the
arbitrary tools/platform/technology project splits, each directory is
versioned in an S-3.4M2-200709210919 directory whose name is computed
based on the time that something was run, rather than anything
detectable. In order to provide my 'direct download links' I had to
manually grep for the random-date-string in order to update my links
each time a new drop was released.

Not only that, but eclipse.org doesn't provide any really useful way
of finding out what plugins are hosted, or even where. It also ends up
wasting a *heap* of bandwith; the -win32-win32-, -win32-wpf-,
-linux-gtk-, -linux-motif-, combos are all pretty much 95%+ the same,
with only a few native bits different from each of them. There's no
'one' place to download the individual plugins.

Even the update sites aren't helpful. You can't browse to them, and
even when you do (e.g. via a mirror) it's not clear as to which plugin
is stored in which location.

Lastly, there's no metadata available. One of the things which will
have been observed from the P2 work is that it is necessary to have a
(small) downloadable metadata file that can be downloaded without
needing to download the bits that the metadata refers to. In Maven,
this is the POM, and without it, a maven repo is just an HTTP store.

Whether the P2 work can be leveraged to have a Maven->P2 link (or vice
versa) is something that could be investigated, but the problem is
that if you rely on e.g. additions to the maven repository then they
are either done sporadically or erroneously. In the case of Eclipse,
several (different) people have uploaded variants of some of the core
Eclipse plugins, but then never get updated as newer versions are
available. If this process were baked into the Eclipse release
process, more people might be tempted to build upon OSGi systems than
they are at the moment.

For those that are interested in building OGSi with Maven, then take a
look at Pax Construct:

There's also Pax Runner, which allows bundles to be launched inside
different OSGi engines, such as Equinox and Felix:

equinox-dev mailing list


equinox-dev mailing list


equinox-dev mailing list