[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [ecf-dev] ECF build v2 -- Build easier

Hi ECF team,
It became a fairly long answer to a short question.

Markus Alexander Kuppe wrote:

4) Is Buckminster going to be replaced by p2?

During the con call the question came up if Buckminster's functionality
of resolving/materializing build dependencies is going to be replaced by
p2 in which case we might not wanna invest on Buckminster. I will ask
the Buckminster team to respond to this question.

The very short answer is yes, to some extent, but P2 does not address source repositories or the build aspect.

The longer answer is that much of this has already happened. Buckminster leverages P2 to it's full extent and is capable of reading P2 update sites, materialize into a target platform definition of type "Software Site", publish to a P2 repository, and also upgrade itself using P2 (the headless install command).

So what has been replaced in 3.5?

1. Buckminster Version + VersionType is now P2 Omni Version
We have been working very closely with the P2 team. Henrik Lindberg (co lead of the Buckminster project) and I desgined the new Omni Version and my implementation of it has replaced the old OSGi Version and VersionRange in the P2 code base. This enables the P2 SAT4J resolver to go beyond OSGi and to store non OSGi IU's and artifacts in P2 repositories. The main reason for us to do this was that we didnt' want to lock Buckminster to the OSGi versioning scheme while moving over to P2 (Henrik also made some major refactorings to the P2 code base to make it more consistent with respect to error handling. He is now a P2 committer. A fact that of course helps us in our continued effort to integrate).


2. Target Platform Management
Target platforms can now be setup by importing a PDE Target Platform Definition (an XML file with .target suffix). Such a definition may well contain a "Software Site" which in essence is a P2 provisioned target platform. A number of headless commands was added to Buckminster to accommodate this.


Buckminsters "eclipse.import" reader is now based on P2 and a new materializer named "p2" has replaced the old "targetPlatform" materializer. Used in combination they can set up a target platform from a number of repositories. The P2 engine is of course in charge of provisioning.

3. Buckminster installer
Is just a very thin layer on top of the P2 director. The reason we retain this layer is for backward compatibility. I submitted a new director application to P2 and it can perform all the tasks that the installer can. Incidentally, we provide a headless packaging of the P2 director on our download page. It's needed when configuring a headless Buckminster.


In addition to these replacements, Buckminster has also started the development of a repository aggregator. It is intended as the successor of the "Galileo Builder" and is capable of slicing and dicing from different sources in order to create a new local repository. It's all based on P2 of course and if you want to take a look, go here: http://wiki.eclipse.org/Getting_Started_With_Aggregator_(Buckminster)

What cannot be replaced?

P2 is designed to deploy runtimes and binary target platforms. The SAT4J resolver is only used when doing the former. When a target platform is provisioned, P2 uses a "Permissive Slicer". It performs basically the same job as the SAT4J resolver but it is more agile (forgiving) and can bypass undesired constraints. For instance, it's very common to use a target platform that is set up to handle any combination of os/ws/arch. The P2 SAT4J based installer is not capable of installing such a thing so P2 provides an extra tool named "repo2runnable" that can convert an artifact repository into a target platform.

The problem with both the SAT4J resolver and the Slicer is that both must have access to all potential candidates in order to do their job. In contrast to this, Buckminsters resolver only needs a top component. It performs its resolution transitively in a step wise manner without knowing all candidates from start. I expect to see improvements on how P2 uses the SAT4J resolver so that its usage can be broadened and cover this case too.

What about P2 and source code repositories? We have given this a lot of though and concluded that using P2 to "install" source will not be easy (if even feasible). Source is moving parts and conceptually, the IU (Installable Unit, the foundation for everything in p2) is immutable. The source of a feature and/or bundle often has incomplete meta-data. You see versions like 0.0.0 in a feature (waiting to be replaced during a build) and versions like 1.2.3.qualifier in the bundles (the qualifier part is replaced during builds). Even if we could do a half decent job of creating meta-data from a set of sources, the task of maintaining it would not be trivial.

Buckminsters dependency resolver is designed to cope with sources and moving parts. It creates its meta-data from the source, on the fly, as needed. So dependency resolution will continue to be performed by Buckminster as we move forward.

The Buckminster CSPEC in relation to a P2 Installable Unit:
Are they the same? In essence, the answer is yes. Going forward, I see the CSPEC become an extension to the IU. In addition to what's covered by the IU (runtime provisioning), the CSPEC also covers build orchestration through its concept of actions, groups, and artifacts. The PDE build team has expressed a lot of interest in a common build infrastructure where everything is modeled and if all goes according to plan, we'll work together with them to create a unified build model for Eclipse. When that happens, our current CSPEC will eventually become obsolete.


Other Buckminster artifacts:
P2 has no concept of materializing from different types of sources (Buckminster can use Maven repositories, PDE style releng map files, CVS, SVN, or Perforce repositories, even web pages with links scanned for patterns, etc.). I don't see P2 moving in a direction where it will address any of that. So the concept of a CQUERY that controls that kind of advanced resolution in combination with an MSPEC that specifies where things should go are also things that will not be replaced by P2.


An extensive effort has gone into writing Buckminster documentation (thanks to Henrik Lindberg) so you can probably get a lot of answers there. You'll find it here:

http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/doc/BuckyBook.pdf

Kind Regards,
Thomas Hallgren