Bug 257004 - Support generating .build files for Galileo contribution
Summary: Support generating .build files for Galileo contribution
Status: RESOLVED FIXED
Alias: None
Product: Modeling
Classification: Modeling
Component: Cross-Project (show other bugs)
Version: 2009-Galileo   Edit
Hardware: PC Linux
: P3 critical (vote)
Target Milestone: 2009-Galileo   Edit
Assignee: Nick Boldt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 259331
  Show dependency tree
 
Reported: 2008-11-30 19:15 EST by Nick Boldt CLA
Modified: 2008-12-22 11:36 EST (History)
14 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2008-11-30 19:15:13 EST
+ promoteToEclipse.sh must support both Ganymede .sc file and new Galileo .build file formats [1]

+ -coordsite galileo will generate the new file

+ -coordsite ganymede will continue to generate the old format (until after SR2 is declared)

+ promo.php must pass galileo/ganymede selection to promoteToEclipse.sh in order to generate the correct metadata contribution.

--

Due to the sensitive nature of .build files and their interconnection to other files (eg., galileo.build), this may need to be done using a headless invocation of the DSL toolkit (if possible?), or by simply modifying existing files to change version numbers.

Even better would be if the <features id="org.eclipse.gmf.sdk" version="2.2.0.v20080917-1925-7A7A3AImms9uVK-KldWYTMMy9wOh"/> attributes could read from external properties files, eg., gmf.build.properties:

org.eclipse.gmf.sdk.version=2.2.0.v20080917-1925-7A7A3AImms9uVK-KldWYTMMy9wOh

and then, in gmf.build:

<features id="org.eclipse.gmf.sdk" version="${org.eclipse.gmf.sdk.version}"/>

Rich, is that possible? Or is there an easier way to feed feature versions into these files?

[1] http://wiki.eclipse.org/Galileo/Build
Comment 1 Richard Gronback CLA 2008-12-01 06:23:38 EST
The individual .build files should be quite static following initial creation, with the exception of the feature version.  I guess the easiest approach would be to create a copy and use it as a template in the build with a token to be replaced as you suggest.

Another approach would be to use QVT to update the model by passing the new version as a configuration property.  This would require having a DSL Toolkit install available on the builder, as you mention.

Technically, there's no reason to update the feature version at all, as p2 will install the latest version it finds in the repository.  But, I figure that would mean lots of broken builds ;)  Another approach would be to grab the site manifest and update each feature version with a QVT prior to generation.  Lots of approaches to consider.
Comment 2 Nick Boldt CLA 2008-12-01 10:11:04 EST
(In reply to comment #1)
> The individual .build files should be quite static following initial creation,
> with the exception of the feature version.  I guess the easiest approach would
> be to create a copy and use it as a template in the build with a token to be
> replaced as you suggest.
> 
> Another approach would be to use QVT to update the model by passing the new
> version as a configuration property.  This would require having a DSL Toolkit
> install available on the builder, as you mention.
> 
> Technically, there's no reason to update the feature version at all, as p2 will
> install the latest version it finds in the repository.  But, I figure that
> would mean lots of broken builds ;)  Another approach would be to grab the site
> manifest and update each feature version with a QVT prior to generation.  Lots
> of approaches to consider.

If the file can be static and not need to feed in the bleeding edge feature version, so much the better. Then I don't need to incorporate any generation into the publishing step, and there's less chance of the file being broken by passing in the wrong feature or too many/few features. 



Comment 3 Richard Gronback CLA 2008-12-01 10:28:41 EST
Agreed, but that "grabs the latest version" feature is an assumption on my part.  I just tried it once and found that it seemingly installed the latest one if found.

More to come, but for now I'd suggest sticking to having component/project leads update the files, at least until M4+.
Comment 4 Nick Boldt CLA 2008-12-01 18:14:22 EST
There's another wrinkle here:

For Modeling, the latest feature may be on the /interim/ site rather than the /milestone/ site, which means that either both the feature version and the repo URL need to be updated for each new build, or if the file contains both repos, the reference to the repo on which the latest feature can be found needs to switch from //@repositories.0 to //@repositories.1 and vice versa.

In other words, if generating this file, we need either @@version@@ and @@repoReference@@:

<?xml version="1.0" encoding="ASCII"?>
<build:Contribution xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:build="http://www.eclipse.org/amalgam/2008/build" label="GMF">
  <contacts name="Richard Gronback" email="richard.gronback@borland.com"/>
  <features id="org.eclipse.gmf.sdk" version="@@version@@" repo="@@repoReference@@">
    <category href="galileo.build#//@categories.0"/>
  </features>
  <repositories location="http://download.eclipse.org/modeling/gmf/updates/milestones/" label="GMF milestones"/>
  <repositories location="http://download.eclipse.org/modeling/gmf/updates/interim/" label="GMF interim"/>
</build:Contribution>

or @@version@@ and @@repoName@@:

<?xml version="1.0" encoding="ASCII"?>
<build:Contribution xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:build="http://www.eclipse.org/amalgam/2008/build" label="GMF">
  <contacts name="Richard Gronback" email="richard.gronback@borland.com"/>
  <features id="org.eclipse.gmf.sdk" version="@@version@@" repo="//@repositories.0">
    <category href="galileo.build#//@categories.0"/>
  </features>
  <repositories location="http://download.eclipse.org/modeling/gmf/updates/@@reponame@@/" label="GMF @@reponame@@"/>
</build:Contribution>

And, of course, it'd be ideal if the list of features (including "enabling features") and contact info was fed from an outside source (ie., generated dynamically) so that this file could simply be created from information already in CVS.
Comment 5 Richard Gronback CLA 2008-12-01 18:56:16 EST
For Galileo, I'd expect the vast majority of contributions to be milestone builds, even if that means building an "a" version.  I'd leave the wrinkle of multiple repository entries for those doing manual updates or to those who have nothing better to do than write complex update/generate scripts.  GMF only has two because I was testing something.
Comment 6 Nick Boldt CLA 2008-12-01 20:36:46 EST
(In reply to comment #5)
> For Galileo, I'd expect the vast majority of contributions to be milestone
> builds, even if that means building an "a" version. 

Didn't doing that last year lead to problems? I'd have thought we wanted to encourage more I build contributions than the once-every-6-weeks milestones...

> nothing better to do than write complex update/generate scripts.  GMF only has
> two because I was testing something.

Complex? Hardly. This is just the 4th in a series of annually-changing xml input files for the aggregate builder -- the format may change, but the data's been the same for at least the last year or so.

I can whip up something in minutes, provided the schema for the xml is final. Is that a safe assumption?
Comment 7 Richard Gronback CLA 2008-12-01 20:42:45 EST
It's pretty stable, but we call 'em "models" not schemas ;).  I didn't mean your approach was complex.
Comment 8 Nick Boldt CLA 2008-12-20 03:00:03 EST
As discussed via email, this is now a showstopper for M4 (bug 259331). Escalating severity.
Comment 9 Nick Boldt CLA 2008-12-21 18:17:29 EST
promoteToEclipse.sh now supports generating .build files for Galileo. Note the following caveats:

a) When created by hand, the Modeling project's .build files contributed a single feature to the site -- the SDK feature. However, for Ganymede, many projects contributed runtime + SDK + other Enabling Features. The generator in promoteToEclipse.sh will produce a list of all these features again (based on the patterns in your promoteToEclipse.*.properties files), but will NOT update galileo.build for you, only your project-subproject.build file.

b) You must manually update galileo.build if you want your "Enabling Features" to be listed in either the Modeling (galileo.build#//@categories.0) or Enabling (galileo.build#//@categories.8) categories; otherwise they'll end up in "Uncategorized". If not using an EMF-based tool to edit the file, just add entries to categories in the order they appear in your project-subproject.build file. Use emf-emf.build's 2 modeling and 6 enabling features as an example.

c) To contribute builds to Galileo, you can either:

i) update your promoteToEclipse.*.properties file to set the following values, so all promoted builds will be contributed by default:
  coordsite=1
  coordsiteName=galileo
ii) or, pass in -coordsite galileo via the commandline / crontab
iii) or, use the checkbox on the promo.php page

If you want to update your M4 contribution using this script instead of by hand, do something like this:

cd /home/www-data/build/modeling/scripts; ./promoteToEclipse.sh -d :pserver:anonymous@dev.eclipse.org:/cvsroot/modeling \
                               -c org.eclipse.emf/org.eclipse.emf.releng/promoteToEclipse.emf.properties,HEAD -coordsiteonly galileo -branch 2.5.0 -buildID S200812151800 -email "nickboldt+bugzilla@gmail.com"

If you have any problems, please reopen this bug & report them here. 

PMCs, please pass this information on to your subproject leads.
Comment 10 Richard Gronback CLA 2008-12-21 18:51:02 EST
Note that the Planning Council determines the categories for the simultaneous release: http://wiki.eclipse.org/Planning_Council/Oct_27_2008

There is no "Enabling Features" category in Galileo.
Comment 11 Nick Boldt CLA 2008-12-21 20:40:19 EST
(In reply to comment #10)
> Note that the Planning Council determines the categories for the simultaneous
> release: http://wiki.eclipse.org/Planning_Council/Oct_27_2008
> There is no "Enabling Features" category in Galileo.

OK, I'll move them to "Tools & Goodies". Enabling makes more sense, but if the PC has spoken, then so mote it be.
Comment 12 Richard Gronback CLA 2008-12-21 20:45:07 EST
Actually, anything not defined by the PC should just appear in the Uncategorized list.  We'll certainly review the categories and their contents once M4 is settled. 
Comment 13 Nick Boldt CLA 2008-12-21 20:51:06 EST
I've moved runtime features to Tools & Goodies & omitted all the Enabling Features from *.build. The Modeling category in galileo.build will now contain only SDK features. 
Comment 14 Richard Gronback CLA 2008-12-22 05:50:42 EST
(In reply to comment #13)
> I've moved runtime features to Tools & Goodies 

That is also a PC-defined category and should not be used for this purpose.

Comment 15 Nick Boldt CLA 2008-12-22 10:42:44 EST
(In reply to comment #14)
> (In reply to comment #13)
> > I've moved runtime features to Tools & Goodies 
> That is also a PC-defined category and should not be used for this purpose.

Removed. Generator now only adds .sdk feature. If a Modeling project, it's assigned to that category; if not, it gets no category. (I presume this means GEF will be an "Unassigned" feature.)
Comment 16 Richard Gronback CLA 2008-12-22 11:36:50 EST
(In reply to comment #15)
> I presume this means GEF will be an "Unassigned" feature.)

Correct