Comment 1 Andrew Overholt 2008-11-14 16:53:26 EST

ant-ify buildUpdateSite.sh in order to combine the steps into one big process.  

these extra build steps should be toggleable via build.properties so that they are optional (based on build type?)

Comment 2 Mickael Istria 2009-02-25 09:14:50 EST

An interesting solution could be to have one build.properties for each build type. Then the right properties file would be read during build according to the build type.
Having a common.properties or default.propertise file for properties that are common to all buils could also be useful.

That way, the common builder wouldn't be responsible of choosing whether some steps have to be done, it would only have to check in the build-type specific file whether they are required.

A "properties" folder could be used, containing
* common.properties
* R_build.properties
* M_build.properties
* S_build.properties
* I_build.properties
* N_build.properties
* default.properties

Ant would then load properties according to the build type
<property file="${propertiesFolder}/common.properties" />
<property file="${propertiesFolder}/${buildType}_build.properties" />
<property file="${propertiesFolder}/default.properties" />

Once this is done, builder won't have to check buildType anymore.

Comment 3 Nick Boldt 2009-02-25 11:02:46 EST

We already have lots of properties files -- do we really want more?

project.releng/build.properties
project.releng/testing.properties
common.releng/server.properties
common.releng/build.properties
common.releng/../testing.properties
test.properties (generated during testing)
build.cfg (aggregate of commandline options and all of the above except test.properties)

Ad the moment, the only difference between the types is that single letter, and that signing is skipped for N builds in the interest of speed. 

Personally, I would prefer to enforce rules such as "N builds do less" than to have to support up to 6 different versions of the same file just for the sake of changing the list of build.steps per build.

--

What we could do, instead, is ensure that build.steps can be set via Hudson parameter (or commandline, or ant property) so individual builds can do less if desired. 

(if ${BUILD.STEPS}, use that; else use default set in project.releng/build.properties)

Would that work?

Comment 4 Mickael Istria 2009-02-25 12:36:32 EST

Having those properties set from Hudson is indeed a better idea!

But the example I had in mind was that each build type should have its specific update site and p2 repo (release-update site, stable-update site, integration...).

Or probably it has already be decided that there would be one update/p2 site per build?

I don't know what is the best practice: having one update/p2 site per build type or having one new site for each build...

Comment 5 Nick Boldt 2009-02-26 02:52:05 EST

I'm thinking one archived update site/p2 repo per build, plus the option to publish that site unpacked somewhere (and replace any existing site already there), so it's only ever a single update at a time.

This would mean you could update or install to get the latest; for older releases, you could download the repo and install from that offline. Much, much simpler than the nightmare that generating and merging multiple sites (in Modeling) has become.

Not sure yet how to address multiple branches of unpacked sites. I guess if the user specifies a target path (tools/gef/updates/e35/) then they could publish there. If they wanted to produce more than one site within the branch (milestones, interim, releases) then it'd be up to the user to decide how many and where).

So we may need a few properties, such as:

update.site.path.34=/tools/gef/updates/e34/ (for R and maintenance)
update.site.path.35=/tools/gef/updates/e35/ (for R and maintenance)
update.site.path.35S=/tools/gef/updates/e35/stable/ (stable builds)
update.site.path.35W=/tools/gef/updates/e35/weekly/ (everything else)

Comment 6 Nick Boldt 2009-03-18 21:01:47 EDT

First phase done: generation of a zipped p2 repo / update site.

I've added a new "buildUpdate" task added to list of build.steps, and enabled it by default.

<property name="build.steps" value="buildZips,buildTests,buildUpdate,generateDigests,test,publish,cleanup" />

Projects already using Athena to build will need to add this step to their list after buildZips or buildTests and before generateDigests.

A site.xml is generated containing a single category in which the features are listed (workaround for https://bugs.eclipse.org/bugs/show_bug.cgi?id=269226); site contains p2 metadata, site.xml, and feature/plugins jars (not packed jars due to https://bugs.eclipse.org/bugs/show_bug.cgi?id=269199)

Additional todos include adding support for associate sites, for mirrorURL, etc.

Comment 7 Nick Boldt 2009-04-21 15:35:19 EDT

buildZips is now optional (bug 272403).

Use of a user-defined site.xml (in order to support assoc sites, mirrorURL, etc. is covered in bug 272991.