Update, Updated
Over the last couple of weeks, I’ve been rebuilding the way the Modeling Project’s update sites work. There are a number of reasons for this:
- To provide a Milestone (S) update site, separate from Interim builds (I & M), in accordance with what GMF & GEF used to provide before they adopted my build system.
- To reduce the disk footprint that has over the years gone from reasonable to insane, and make it self-cleaning.
- To provide site digests for improved performance.
The upshot is that I now have a system that incrementally adds new builds’ jars to the site, cleans out any obsolete jars (ie., if replaced by newer version) and regenerates both the site.xml and digest.zip for the site. Instead of 7.9G, the 7×3 sites (EMF, EMFT, MDT, M2T, M2M, GEF, GMF) now occupy 1.3G.
How does it work? Read on. Don’t care? Skip here.
It all starts with either:
- an SDK zip (or a few zips, if you produce more than one like GMF),
- a “Master” zip (the result of collecting all your build’s features and plugins into a single zip, then sending it to build.eclipse.org for signing), or
- an Update zip (much like the SDK, but including a site.xml file and with the jars in plugins/ and features/ instead of in eclipse/plugins/ and eclipse/features/).
From there, an individual “sitelet” (category.xml) is created for those jars. Unjarred features/plugins are jarred. The sitelet - a single update site category - is combined with other local sitelets & their associated jars into one composite update site. A digest is produced.
Rather than just pushing that whole site from the build server to the remote public server, I use a few tricks to avoid having to upload the same stuff over and over.
- If the new site, which I’ll call the nth release, or rN, is identical to the zeroth (current) release, r0, then the process stops right there, as there’s no point publishing the same bits again.
- if rN != r0, we cycle the releases so we only ever have three of the same build’s branch (2.1.0) and type (S). r2 is deleted, r1 becomes r2, r0 becomes r1, and rN becomes r0.
- Depending on the type of site, we will then aggregate 1, 2, or 3 of these individual sites. For R, we only need the latest; for S we want the last two milestones; for the others, we keep 3.
- This aggregate site is used to produce a “clean” list of all the jars that SHOULD be present for the current crop of builds.
- A zip is produced containing the new plugin & feature jars, all the current individual category.xml files, a couple of shell scripts (buildUpdateSiteXML.sh, buildUpdateSiteDigest.sh) & the jar list. After pushing this zip to build.eclipse.org, the zip is unpacked and its contents combined with the current update site of the same type.
The new site’s contents are collected to produce a “dirty” list of jars. This list is compared to the “clean” list, and anything no longer needed is purged from the site.
The category.xml files are combined into a single site.xml, and from that the digest.zip is created.
If you’ve been installing any of the Modeling projects from update sites in the past, please try out the new version and compare its performance with your old experience. Feedback to bug 212203. See also the following p2-related issues. There’s not a lot of time left in this cycle, so vote for your faves!
- 200380: [prov] [solver] Improve detection of incompatible solution
- 223655: Cannot install from an update site that does not use site.xml
- 228145: p2 is inventing invalid jarname suffixes [SOLVED]
- 228310: Installing a newer version of a pre-installed feature should proceed as an update
- 229679: p2 does not install dependencies properly
Test early, test often!
Posted April 30th, 2008 by in category: build, ganymede, p2, pde, update manager
You can skip to the end and leave a response. Pinging is currently not allowed.
Leave a Reply
You must be logged in using your Eclipse Bugzilla account to post a comment.
