[stp-dev] [Fwd: Round 1 - Build System Proposal]

[Adrian Skehill <adrian.skehill@xxxxxxxx>]

Hi Naci,

Here's the original proposal that I sent out to the news group. We can 
chat about this when you have time.

I should be free at the end of this week to work on the build system.

Adrian.

-------- Original Message --------
Subject: Round 1 - Build System Proposal
Date: Wed, 18 Jan 2006 15:29:19 +0000
From: Adrian Skehill <adrian.skehill@xxxxxxxx>
Organization: EclipseCorner
Newsgroups: eclipse.stp

This is something that I've put together to get the ball rolling on the
build system. It's based primarily on the build system that we've been
using to build our eclipse based plugins within IONA for the last year
or so.


The highlights of our build system are as follows:
  * Ant based
  * Provides a complete automated mechanism to build eclipse based
plugins as part of our product build cycle
  * Does this build across UNIX based OS's (Linux) and Windows
  * Leverages existing ant based build systems to build components that
are required by our tools
  * Provides automated testing infrastructure, uses Eclipse unit test
infrastructure and interacts with Segues SilkTest for system level testing.
  * Integrates with JCoverage to provide coverage figures on a regular 
basis
  * Provide hooks for CruiseControl to do automated builds following
each update of the source base.

One of the bigger limitations of this system right now is that there is
no integration with CVS. We've built this within our own development
environment of ClearCase so I suspect we may have to do some work here
to get it to fully support CVS builds. I've only just started looking at
CVS in recent weeks, but I'm guessing that it should not be too much
work to get it up and running in this environment. Famous last words!

So I've attached a very rough diagram outlining our build system
(buildsytem.jpg). As you  can see from this, we have one root build.xml
file, which includes a bunch of ant include files stored in the ant-inc
directory a few levels deeper. We've also created a number of ant tasks
specifically for build eclipse code which offer us the ability to parse
the MANIFEST.MF file to determine the correct build order, correct jar
names and plugin names so that the build artefact's are identical to
what is produced if using the Eclipse IDE. I was unable to get
ant4eclipse up and running with Eclipse 3.1 based (OSGI) plugins so went
a quick simple solution in this respect.

Evey plugin has a corresponding .tests fragment which obviously tests
the plugin. The test suite takes each plugin / fragment pair and kicks
off Eclipse to invoke those tests only. Eclipse is shutdown and
re-started for each plugin. When all plugins have been tested, a JUNIT
style report is produced.

In terms of how we bring in functionality from third party (that means
non IONA for now!) and IONA components, we created two special plugins
that are merely facade plugins. These plugins are auto generated using
XSLT which describes which jars are needed and brings the jars directly
into the plugins lib directory. Finally, the ant build system itself
needs to be bootstrapped to identify where the eclipse installation is
that can be used to build and test the plugins. At the moment we have a
staged copy of eclipse living within our development environment which
we use build the plugins. This contains full plugins for running on
Windows / Linux together with CDT and EMF plugins (all of which our
tools use). It also contains the eclipse automated testing plugins.

As an aside site.xml and feature.xml are also generated when required so
that any updates we need to do can be easily done.

When initially designing this build system, there were a few things that
I was striving for:
* I didn't want to have to exclusively maintain this, it should be
straightforward enough that every contributor could easily update it
when required.
  * It was completely automated and could run reliably each night and
produce reports without any intervention.
  * Integrated well with Eclipse so changes to plugins within the IDE
required as little as possible updates to the ant build system
  * Was simple... darn said that already!


So in terms of improving this for the STP project I'd have the following
suggestions to make.
  * Keep it nice and simple and stay with ant (I personally don't think
we should move to Maven to keep complexity down, but then again I've
only ever had fleeting glances at Maven in operation). But it probably
makes sense to leave this open to discussion.
  * Keep the directory structure intact like as follows only allow
multiple components under the root (e.g. framework, service_creation)
each which would define their own set of plugins
  * I'm not sure if we would need to have a build.xml present in each
directory or not, comments? Might make sense though.
  * Shared components, i.e. access to the test suite, coverage etc.
would all be defined in the ant-inc directory.

Let me know what you think.

I'll be online at the IRC meeting later on this afternoon if anybody
needs to ask more detailed questions.

Regards,
Adrian Skehill.

Artix UI Team Lead,
IONA Technologies.