Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[recommenders-dev] Mangement of third party dependencies
hi,

yesterday we had some trouble with updating our dependency to SLF4J from 1.6.5 to 1.7.2. The update was required because 1.6.5 caused some unexpected trouble with the Kepler release train [1]. After quite a few problems with the current system, Andreas finally could fix this issue (Thanks Andreas!) but the demand for changing the way how we manage our 3rd part dependencies is bigger than ever.


The current process of adding a 3rd party lib to our build is as follows:

  • In our 3rd git repo, add a new project with the bundle's name (e.g., com.google.guava).
  • In this project, add the class files as bin folders to the project and classpath (only class files, no sources.)
  • Update or create the required OSGI metadata
  • Add the plugin to the 3rd feature
  • Add and update the pom files
  • Push to 3rd git repo. Let Jenkins pick up the changes and publish the new 3rd p2 repo.
  • Done.

At the moment we build this site with maven tycho 0.13 - which fails on some machines and thus makes the release quite painful. So there is a need to update things.


So here is my proposal.

First, we update the build to Tycho 0.17. Then we add a manually maintained "local as-is" p2 repository which contains ready to use osgified libs only. New bundles are just copied into and the p2 repository metadata is regenerated with the p2 publisher.

Then there is a git repository that contains the projects that need to be build from scratch (already exists). This includes all our not-yet osgified or patched libs as well as all features we need for drive the Eclipse Recommenders builds. It also contains the target platform definition that makes all required p2 update sites know to the tycho build. When the build completes, the resulting p2 update site is published to a certain download folder with the current timestamp and build type in it ("IyyyMMdd-hhmm" usually):




Implications:
Every time we update our dependencies, we have to update the .target files to reflect this change. This isn't great but the alternative of having a single "latest" URL only has proven to work badly in the past.

One question to answer is, how many libs do we want to build with Tycho? Wouldn't it be easier to just patch the zip files directly (i.e, unzip, patch the manifest, zip)? In general I think this would be sufficient in most cases. So, the 3rd git repo may only contains those libs that actually have to be patched and build in a more complicated way. We may consider to use submodules to keep the patched bundles separate from the target and feature definitions.

Andreas was asking whether we should use fixed version in our features. In general, I'd say yes but this will cause a lot of extra trouble. I don't know out of my head whether we have to specify the qualifier in the feature.xml too to make the build work. But when we we have to specify the qualifier, every change in a qualifier will break our build. This will get annoying pretty soon. I think we should test (or someone can already tell me?) whether the qualifier is required.

To summarize,
the core changes over the current approach I propose ATM are  (i) the as-is p2 repository (marked orange in the illustration) and (ii) the publishing to particular download folders.

Andreas, what do you think?

Best,
Marcel







Back to the top