Bug 436736 - Prep for 4.5 (Mars) and 4.4 maintenance builds
Summary: Prep for 4.5 (Mars) and 4.4 maintenance builds
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 4.4   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 4.4.1   Edit
Assignee: David Williams CLA
QA Contact:
URL:
Whiteboard:
Keywords: info
Depends on: 437787 437820 438985 439154 439189 439192
Blocks:
  Show dependency tree
 
Reported: 2014-06-05 13:45 EDT by David Williams CLA
Modified: 2015-06-09 14:13 EDT (History)
12 users (show)

See Also:


Attachments
Build id: I20140701-0800 is said to be "Mars (4.4)" (37.37 KB, image/png)
2014-07-02 06:51 EDT, Szymon Ptaszkiewicz CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2014-06-05 13:45:59 EDT
This is just a general bug to make notes of what I (we) need to do to build after current development cycle ... and when we'd like to start ... for things I recall off the top of my head ...

= = = = For Mars (4.5): 

- a few bootstrap/cronjobs have to be changed to specify "4.5.0" instead of "4.4.0". 

- the "streams/repositories.txt stays as it is, in master, 

- Many "parent poms" have to change from version 4.4.0 to 4.5.0 -- and this has to be "coordinated" (technically, we could probably use 4.4.0 for a few weeks, as long as nothing "published" or "deployed" with "4.4.0" or it would be wrong.
[Thanh, do you recall details/impacts of "when to update"?].  

- the API reference URL changed to point to 4.4 Release (and the "M6 checks" disabled, I believe by simply not providing a URL) 

- the build schedule changed back to 8 PM nightlies, Tuesday 8 AM I-builds. 

- The <releaseName> in parent pom changed to "Mars". 

= = = = For maintenance, 

- branch aggregator to R4_4_maintenance

- a few bootstrap/cronjobs have to be changed to specify "4.4.1" and R4_4_maintenance and buildType=M (instead of I). 

- in the past, we've left "parent poms" as "point zero" even though true maven practice would be to update them to 4.4.1 ... but 4.4.0 seemed to work ok for us. 

- the product Id versions have to change to 4.4.1 (in pom, and manifest) to be displayed correctly in about.box. 

- the "streams/repositories.txt file will need to be updated if/when projects branch ... or their 4.4.0 tags used if "no changes"

- the M-build repo (used for "comparator" has to be "primed" with last I-build of 4.4-I-builds. 

- the comparator URL in aggregator needs to be changed to point to 4.4-M-builds. 

- ep44M-unit-* test jobs need to be created (technically, not strictly necessary, but things work better that way, since a "sequence of logs or results" etc., are all for same branch ... and occasional advantage that an I-build test can be running at the same time an M-build test is). 

- the API reference URL changed to point to 4.4 Release (and the "M6 checks" disabled, I believe by simply not providing a URL) 

- the build schedule changed to have M-builds at 10 AM Tuesdays (I believe is what we've decided in past is what worked best ... i.e. have I and M builds on same day). 

- the <releaseName> in parent pom changed to "Luna SR1"

= = = = = = = = = = = 

As for "when to start" ... that's mostly up "the team". I believe in the past we've started about a week or 10 days before current "release", with the possible complication that if we need an emergency re-build, we need to use care and coordinate several things with "side branches" and make sure to use "static" versions of the repos that are not intended to change (preferably tags, but if nothing else, could always use the "I-build" tag from released build, if we've not yet made official R4_4 tags (which we typically do not do until after the formal release). 

But, we only need to rush, if "the team" really needs it. I don't think anyone (i.e. me) wants the stress of "more to do" if there is no urgent reason to -- in which case we'd wait until after June 25th. 

= = = = = = = = = = = 

I'm sure there's more I've forgotten ... this is to start the discussion and planning.
Comment 1 David Williams CLA 2014-06-10 13:11:16 EDT
small reminder to self ... I've turned off my "clean up build machine" cron jobs ... to make sure we don't remove the build we later want to promote as "final" ... so I'll need to remember to turn those back on. (after promotion).
Comment 2 Markus Keller CLA 2014-06-11 12:20:34 EDT
I don't think we should complicate releng by restarting regular builds before Luna is declared.

Individual components can just create a new branch off of master to start Mars M1 work (e.g. marsTemp, as I just did for JDT UI). Once Luna is declared, the branch can be merged into master (use a fast-forward merge if possible, but don't rebase, since that would make links to commits in the marsTemp branch invalid).
Comment 3 Thomas Watson CLA 2014-06-12 09:41:26 EDT
I agree with Markus.  No need to start Mars builds until after Luna is delivered.
Comment 4 Dani Megert CLA 2014-06-12 17:53:55 EDT
(In reply to Thomas Watson from comment #3)
> I agree with Markus.  No need to start Mars builds until after Luna is
> delivered.

+1.
Comment 5 Paul Webster CLA 2014-06-19 17:11:51 EDT
(In reply to David Williams from comment #0)
> = = = = For Mars (4.5): 
> 
> - Many "parent poms" have to change from version 4.4.0 to 4.5.0 -- and this
> has to be "coordinated" (technically, we could probably use 4.4.0 for a few
> weeks, as long as nothing "published" or "deployed" with "4.4.0" or it would
> be wrong.

Component leads should add themselves to Bug 437787 so we can co-ordinate the change.

PW
Comment 6 Markus Keller CLA 2014-06-26 14:57:58 EDT
R4_4_maintenance branches have been created for all jdt repos and many platform repos. Missing at this time:

eclipse.platform
eclipse.platform.releng.aggregator
eclipse.platform.resources
eclipse.platform.team

pde

equinox (Maybe want to use R3_10_maintenance? Note that R3_9_maintenance branch and R4_* tags are already out of sync.)


To create top-level branches on git.eclipse.org, you need shell access. Either:

build.eclipse.org$ git config hooks.allowcreatenottopicbranch true
in EGit or local$ <create branch R4_4_maintenance at R4_4 and push it>
build.eclipse.org$ git config --remove-section hooks

Or even easier, create the branch directly on the main repo with:

build.eclipse.org$ git branch R4_4_maintenance R4_4
Comment 7 Paul Webster CLA 2014-06-26 15:30:59 EDT
(In reply to Markus Keller from comment #6)
> R4_4_maintenance branches have been created for all jdt repos and many
> platform repos. Missing at this time:
> 
> eclipse.platform

I've done eclipse.platform.ui/runtime/ua and eclipse.platform.

PW
Comment 8 Curtis Windatt CLA 2014-06-26 15:40:47 EDT
(In reply to Markus Keller from comment #6)
> pde

R4_4_maintenance branches added for the pde repos
Comment 9 David Williams CLA 2014-06-27 02:45:58 EDT
    For the record, one thing I forgot to mention for "mars", but did for "maintenance" is that I had to create new Hudson jobs; ep45N-unit-* (x3) and ep45I-unit-* (x3) since by convention we run each "type" and "stream" in a different job (even though the contents of the "config" is exactly the same ... it just aides book keeping. 

    So tonight's (6/26) tests did not start automatically, but failed since the jobs didn't exist yet. So, I then created the jobs, and started the tests "by hand". 

    Subsequent builds will test to confirm they will still start automatically (without further changes).
Comment 10 David Williams CLA 2014-06-27 02:59:40 EDT
Thanks for creating the branches to use in streams/repositories.txt (once I branch the aggregator) ... and wanted to clarify my initial plan is to set all the projects to their 4.4 tagged versions: R4_4, for maintenance .... primarily just to "test the build" and make sure that works with everything "as it was". 

Then I think it's an open question, and a "team answer" if I should just change them all to use the branch ... or, leave it at the tag, until until I'm told to change it to the branch (that is, leave at tag, until changes are made in the maintenance branch). 

I think part of the answer depends on if anyone things there might be repositories that are not changed at all for SR1 (or, at least, not for a long time) ... I think practically speaking, the only advantage to leaving as tag (if no changes) is then there is less "build tags" in the repository ... which I get the impression some prefer as being less noise ... but ... others might prefer as "extra confirmation" of what code went in which build (even if constantly the same). 

BUT, primarily, my point is if you see me change to use all tags for a few days, "don't panic" ... that's just me making sure that works, before moving on. Once we start maintenance builds "for real", I'm happy to change all to branches, or change to branches only "upon request" ... which ever way the team (or component?) wants it. 

Thanks again,
Comment 11 Szymon Ptaszkiewicz CLA 2014-06-27 07:57:14 EDT
I created R4_4_maintenance branches for eclipse.platform.resources and eclipse.platform.team.
Comment 13 Thomas Watson CLA 2014-06-27 12:46:41 EDT
(In reply to Markus Keller from comment #6)
> equinox (Maybe want to use R3_10_maintenance? Note that R3_9_maintenance
> branch and R4_* tags are already out of sync.)

Note that equinox moved up to be back in sync with eclipse release versions for Luna (4.4).
Comment 14 Szymon Ptaszkiewicz CLA 2014-07-02 06:51:52 EDT
Created attachment 244725 [details]
Build id: I20140701-0800 is said to be "Mars (4.4)"

It seems that we still have a reference to 4.4 in latest I-build. See the attached image for details.
Comment 15 David Williams CLA 2014-07-03 03:02:49 EDT
(In reply to Szymon Ptaszkiewicz from comment #14)
> Created attachment 244725 [details]
> Build id: I20140701-0800 is said to be "Mars (4.4)"
> 
> It seems that we still have a reference to 4.4 in latest I-build. See the
> attached image for details.

Thanks, Szymon. 
This basically comes from some "hard coding" in the branding bundles, specifically for "about.box" (for which we have two, Eclipse SDK, and Platform). For Eclipse SDK, in plugin.properties, says: 

productName=Eclipse SDK
productBlurb=Eclipse SDK\n\
\n\
Version: {1} (4.4)\n\
Build id: {0}\n\

For other branding bundles, they don't influence about.box, but do show up in "list of features" and there it is controlled by 'blurb' in "about.properties" (not 'productBlurb' in plugin.properties) where a typical snippet is similar to: 

blurb=Eclipse Help\n\
\n\
Version: {featureVersion}\n\
Build id: {0}\n\
\n\

I think I'll continue with the "hard coding" for the two product cases, though it would be nice not to, since basically then we have the tail wagging the dog ... i.e. we have to change a file to get the version to show up correct, which requires us to increase the version of the product (where as an increase in version of product should automatically be reflected in about.box). 

I'm sure there is a "pure" solution, that would not require hard coding, but it's kind of mute, since it's a given the two products will be "4.5" and not sure it's worth the effort to spend more than 5 minutes on it ... 

Well ... I will think about it for a day or two. We basically get the 
"Mars" from defining <releaseName> in the parent pom, and then assign that to "localization" variable {1} in about.mappings: 


0=${buildId}
1=${releaseName}

I suppose it'd be easy to define <SDKReleaseNumber> and <PlatformReleaseNumber> so then we'd have explicit control over the "release number" no matter what the "featureVersion" was. 

This would be good for cases where, say, "the Platform" really deserves merely a service field increase, but as a product we'd want it to "match" the SDK? 

Opinions? Hard code? Or use variables?
Comment 16 David Williams CLA 2014-07-03 12:40:21 EDT
This was the main commit to bring R4_4_maintenance "in line" with master (where they should be in line) and prepare to build 4.4 M builds. 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?h=R4_4_maintenance&id=abafcf330b27dcfd1cffb0407f5055f5211dcc4f
Comment 17 David Williams CLA 2014-07-05 10:29:22 EDT
One thing forgotten was the "pre-req" repo where people build "individual bundles" and want to simply pull in "the latest" from previous Eclipse builds, for Eclipse pre-reqs they don't build ... such as see bug 438961. 

These means "Mars" stream should have (and has been) updated as 

    <!-- this eclipse-p2-repo.url is specifically for the build individual bundles profile. -->
    <eclipse-p2-repo.url>http://download.eclipse.org/eclipse/updates/4.5-I-builds</eclipse-p2-repo.url>


And for Luna Service, it should be updated to 4.4-M-build: 

    <!-- this eclipse-p2-repo.url is specifically for the build individual bundles profile. -->
    <eclipse-p2-repo.url>http://download.eclipse.org/eclipse/updates/4.4-M-builds</eclipse-p2-repo.url>

which I just now updated with commit 

0b1c77d4d571768d72398e6df92557799c59ccc7
Comment 18 David Williams CLA 2014-07-05 14:15:38 EDT
(In reply to David Williams from comment #15)
> (In reply to Szymon Ptaszkiewicz from comment #14)
> > Created attachment 244725 [details]
> > Build id: I20140701-0800 is said to be "Mars (4.4)"
> > 
> > It seems that we still have a reference to 4.4 in latest I-build. See the
> > attached image for details.
> 
> Thanks, Szymon. 
> This basically comes from some "hard coding" in the branding bundles,
> specifically for "about.box" (for which we have two, Eclipse SDK, and
> Platform). For Eclipse SDK, in plugin.properties, says: 
> 
> productName=Eclipse SDK
> productBlurb=Eclipse SDK\n\
> \n\
> Version: {1} (4.4)\n\
> Build id: {0}\n\
> 
> For other branding bundles, they don't influence about.box, but do show up
> in "list of features" and there it is controlled by 'blurb' in
> "about.properties" (not 'productBlurb' in plugin.properties) where a typical
> snippet is similar to: 
> 
> blurb=Eclipse Help\n\
> \n\
> Version: {featureVersion}\n\
> Build id: {0}\n\
> \n\
> 
> I think I'll continue with the "hard coding" for the two product cases,
> though it would be nice not to, since basically then we have the tail
> wagging the dog ... i.e. we have to change a file to get the version to show
> up correct, which requires us to increase the version of the product (where
> as an increase in version of product should automatically be reflected in
> about.box). 
> 
> I'm sure there is a "pure" solution, that would not require hard coding, but
> it's kind of mute, since it's a given the two products will be "4.5" and not
> sure it's worth the effort to spend more than 5 minutes on it ... 
> 
> Well ... I will think about it for a day or two. We basically get the 
> "Mars" from defining <releaseName> in the parent pom, and then assign that
> to "localization" variable {1} in about.mappings: 
> 
> 
> 0=${buildId}
> 1=${releaseName}
> 
> I suppose it'd be easy to define <SDKReleaseNumber> and
> <PlatformReleaseNumber> so then we'd have explicit control over the "release
> number" no matter what the "featureVersion" was. 
> 
> This would be good for cases where, say, "the Platform" really deserves
> merely a service field increase, but as a product we'd want it to "match"
> the SDK? 
> 
> Opinions? Hard code? Or use variables?

The more I think about it, the more I know "variables" are the right answer. 
I will add 2 (for now), named slightly differently than I suggested above: 
releaseNumberSDK
releaseNumberPlatform

= = = = = = = 

On a related note, we should change the splash screen to say "Mars" instead of Luna, in the 4.5 stream ... even if it is temporary (and a new splash screen made/contributed later in the cycle.
Comment 19 David Williams CLA 2014-07-05 14:53:06 EDT
(In reply to David Williams from comment #18)
> 
> The more I think about it, the more I know "variables" are the right answer. 
> I will add 2 (for now), named slightly differently than I suggested above: 
> releaseNumberSDK
> releaseNumberPlatform
> 

http://git.eclipse.org/c/platform/eclipse.platform.releng.aggregator.git/commit/?id=8819443e938a31744dfa683864109ed0c09d7ded

http://git.eclipse.org/c/platform/eclipse.platform.git/commit/?id=28397b39c012571991eef1ff846eb2e3af14c60b

These are the commits to effect this change, in master, will still need to make the change in maintenance, so it matches. 

And NOTE: this does not change the need to increment the feature number and branding bundle version ... but technically that is a separate decision and technically would not have to match. For example, if we really did do "only maintenance" in the Platform, then by the usual "versioning rules, it might be only 4.4.100 or something ... if we wanted to be picky. (And, I am not saying we should be ... just saying it's a separate decision).
Comment 20 David Williams CLA 2014-07-06 11:01:39 EDT
> - the "streams/repositories.txt file will need to be 
> updated if/when projects branch ... 
> or their R4_4 tags used if "no changes"

I've decided it would be easier for everyone if I just changed all maintenance streams to R4_4_maintenance whether there has been changes or not ... just so nothing has to be coordinated, changed in future ... the only disadvantage is this puts a lot more "build tags" in a repo that has no changes (which ... isn't all bad either). BUT, if component owner would prefer I use "release tag" so they can avoid the verbose build tags, just say the word. 

I made this change on Sunday, 7/6, so should be "ready to build" our first M-build.
Comment 21 David Williams CLA 2014-07-06 13:14:11 EDT
> - the build schedule changed to have M-builds at 10 AM Tuesdays 
> (I believe is what we've decided in past is what worked best ... 
> i.e. have I and M builds on same day).

Correction, I looked at the build schedule from last year, and it seemed we routinely had M-builds on Wednesday, at 8 AM. So that's the plan I will start with ... but again, if the team wants and agrees on some other schedule I'm fine with what ever the team wants. 

I have "blindly" updated the build schedule for July and August: 
https://www.eclipse.org/eclipse/platform-releng/buildSchedule.html
(and by blindly, I mean I think we'll have M1 before then, but have not scheduled that in).
Comment 22 David Williams CLA 2014-07-06 13:21:28 EDT
FYI, I also added "mars" too the splash screen in master ... though of course I'm not saying that's permanent, or final splash screen for Mars if/when anyone wants to contribute a new one ... just thought it'd help the team "know" they were running the right version. 


So, with all that, I think we are done with this bug. 

If any new issues some up, please open new bugs (and refer back to this one, if it should have been "routine" part of starting a new maintenance branch and new development year). 

One "nice of have" follow-up is, perhaps, to capture some of the routine steps in a "how to" wiki, or something, but not sure it will be all that constant from year to year ... but have marked this bug with "info" keyword, meaning (to me) is it can be used for reference information.