Community
Participate
Working Groups
Just wanted to start a "to do" list of what's needed to do 4.3 (I and N builds) and 4.2 maintenance builds. [I'll also mention there's been some interest in having "official" "Test Builds" that would to through same process as other builds, including tests (though still no reason or request to put on "downloads" server ... not even sure there is much a reason to put N builds on downloads server, except people would have to access using a different URL). So, I just mention this "Test Build" request, in case changes are needed, to keep in mind having a build_type of "T", again).
BTW, current plan is this would all in done "in July" ... or last week of June, after the official release, after all is tagged, etc. For the main build code, not much change, if any, is required. Some of the jobs we use to start build, such as from cronjobs, would take some tweaks, so that "master" would correspond to "4.3" and 4_2_maintenance would correspond to 4.2.1. Technically, we could do all this in 'master' and it'd still work for both streams. We would only need a branch of 'aggregator', technically, once we had different branches in "streams/repositories.txt". Eventually, we'll have to decide on a case-by-case bases which "pre-reqs" get updated in each aggregator stream ... such as if Tycho 0.19.0 is released, will we want to use that in 'maintenance', etc. Some code that "promotes" the builds might need tweaking, too, such as, for example, "4.3.1 builds are prefixed with M, to to 4.3-M-builds repo, etc.
I should (also) re-add some Hudson test jobs that tests 4M separately than 4I builds ... this technically is not required, but, is handy since for several platforms, the tests can overlap and "run at same time" if there are different jobs, but not if one job that handles both.
Just to be clear, in the last week of June (say): master becomes 4.4 R4_3_maintenance becomes 4.3.1 right? PW
(In reply to comment #3) > Just to be clear, in the last week of June (say): > > master becomes 4.4 > R4_3_maintenance becomes 4.3.1 > > right? > Definitely.
At the last status meeting, I was asked to do this "soon", since if an emergency respin was required, there are ways to do that with "side branches" ... it doesn't have to be master. It was requested, though, that I do a test build, using everyone's final Kepler tags I20130605-2000 in repositories.txt to confirm gave same output. I plan do do this test as a "build machine only" test run (i.e. no run through unit tests) and do binary compare on resulting repo and a few packages.
*** Bug 410744 has been marked as a duplicate of this bug. ***
Just to make a note, while I'm waiting for build to finish, even if using the same input (I20130605-2000) produced an identical build "today", it might not, say a few weeks or months from now, since we do not yet use .target files to specify exact versions of all our prereqs (e.g. from emf, ecf, orbit) (tracked in bug 400518). To minimize "danger" of this, we currently use "exact repository" URLs (instead of composites) plus, we even remove "local cache" first so if a repository were to "disappear", we would at least know about it. (and could likely find where it moved to, say as it went from "RC candidate" build to its "released" location). Just making a note of it, so I will be less likely to forget, in case we do have to reproduce a build in a week or two or a month from now.
There are a number of places where we define "the version" in POMs, such as "eclipse-parent" ... <version>4.3.0-SNAPSHOT</version> So, I'm guessing, that should change to 4.4.0 for Luna and 4.3.1 for maintenance. Does this mean EVERY project has to update the places where they refer to "parent pom"? That is a lot of POM changes ... which I assume will have to be carefully timed/coordinated to work. Is there no way to make this more of a "variable"?
(In reply to comment #8) > There are a number of places where we define "the version" in POMs, > such as "eclipse-parent" ... > > <version>4.3.0-SNAPSHOT</version> > > So, I'm guessing, that should change to > 4.4.0 for Luna > and > 4.3.1 for maintenance. > > Does this mean EVERY project has to update the places where they refer to > "parent pom"? That is a lot of POM changes ... which I assume will have to > be carefully timed/coordinated to work. > > Is there no way to make this more of a "variable"? If not a variable, then it also implies everyone needs to make a branch for maintenance, right? Granted, we could live with continuing to use "4.3.0" for a while, I guess ... but doesn't seem good or quite right to do that, since it is going into "maven repo" for others to use?
Adding some other Maven/Tycho experts to CC specifically to request advise on the versioning of "parent poms". Is there a why to make it a variable? Or for children to omit it? Or does everyone literally need to make a branch at the same time to refer to the correct version (4.3.1, vs. 4.4.0). Is there a "best practice" used by projects experienced with maven/tycho? Thanks for any help.
In regular maven projects, a common practice is to upversion all of the POMs (like from 4.3.0-SNAPSHOT to 4.3.0), do the release build, and then upversion all of the POMs to the next version (in our case, 4.4.0-SNAPSHOT) As things stand now, we probably expect to upversion our parent poms/repo poms from 4.3.0-SNAPSHOT to 4.4.0-SNAPSHOT for Luna and 4.3.1-SNAPSHOT for Kepler SR1. Another potential option (I'm less supportive of) would be to continue to use 4.3.0 for 4.3.x (since it's deployed as a snapshot to repo.eclipse.org) ... we need the versions distinguished if we need to build 2 branches at the same time (like 4.3.x and 4.4.0) so we would still have to upversion all of the references to parent in the poms to 4.4.0-SNAPSHOT. To the best of my knowledge, it's not easy to use properties/variables in parent version/artifactId/groupId. This is another perspective on releases: http://blog.xebia.com/2012/09/30/continuous-releasing-of-maven-artifacts/ I'm not really sure how to apply it.
The test build completed as expected. It's here on build machine, if someone wanted to look at anything: http://build.eclipse.org/eclipse/builds/4I/siteDir/eclipse/downloads/drops4/I20130613-1325/ I did a "diff" on the repository features and plugins, and just saw the expected differences ... there are 20 "branding bundles" that get a qualifier of build date, instead of what's changed in CVS. The other 538 bundles had same qualifier and "diff" said they were identical. One oddity was the custom "batch compiler". Both RC4 and the test build had same version, org.eclipse.jdt.core.compiler.batch_3.9.0.v20130604-1421.jar but diff said there was a difference. But, I unzipped them, and did a diff on those directories, and it said all files were identical. So, some oddity in the exact way it's jarred, or something, that doesn't effect the functional content. (Similar to bug 408944?) So, I think we are good to go ... just need to decide what to do about "parent pom" version ... change immediately, or wait a week or two?
I'm fine with waiting the week and a half and updating the pom parent version when Kepler releases for real. AFAIK we're not going to change any of the 3rd party URLs just yet, right? PW
(In reply to comment #8) > There are a number of places where we define "the version" in POMs, > such as "eclipse-parent" ... > > <version>4.3.0-SNAPSHOT</version> > > So, I'm guessing, that should change to > 4.4.0 for Luna > and > 4.3.1 for maintenance. > > Does this mean EVERY project has to update the places where they refer to > "parent pom"? That is a lot of POM changes ... which I assume will have to > be carefully timed/coordinated to work. > > Is there no way to make this more of a "variable"? Unfortunately not. Maven uses this information as metadata so it cannot be a variable as far as I'm aware.
(In reply to comment #12) > > One oddity was the custom "batch compiler". Both RC4 and the test build had > same version, org.eclipse.jdt.core.compiler.batch_3.9.0.v20130604-1421.jar > but diff said there was a difference. But, I unzipped them, and did a diff > on those directories, and it said all files were identical. So, some oddity > in the exact way it's jarred, or something, that doesn't effect the > functional content. > (Similar to bug 408944?) > I looked into this a little deeper (since should not be the same issue as bug 408944) and found I could use "zipinfo" to get detailed information about the zip files' "metadata" (jar files, of course, but zipinfo works there too). I won't print all the data on each jar, but doing a "diff" on those reports showed that what was different was the "time stamp" on each class file. For example, the diff file starts off as 1c1 < Archive: I20130605-2000/repository/plugins/org.eclipse.jdt.core.compiler.batch_3.9.0.v20130604-1421.jar 1830195 551 --- > Archive: I20130613-1325/repository/plugins/org.eclipse.jdt.core.compiler.batch_3.9.0.v20130604-1421.jar 1830195 551 30c30 < file last modified on (DOS date/time): 2013 Jun 5 21:26:42 --- > file last modified on (DOS date/time): 2013 Jun 13 14:51:24 57c57 < file last modified on (DOS date/time): 2013 Jun 5 21:26:42 --- > file last modified on (DOS date/time): 2013 Jun 13 14:51:24 I think this explains why the comparator always "sees them as different" even though nothing has changed ... simply because they have different timestamps on files. This likely doesn't happen for a "normal" (non-custom) plugin because someone (Tycho? Or Compiler?) is smart enough to keep the time stamp of .class file the same as the .java source file. But in the custom case, by design, it has no idea where those .class files come from so they get what ever date they happen to have when copied. ... Yet another good example of why we use the comparator strategy to begin with :) I know that Ant "copy" has a 'preservelastmodified' attribute, but did not see anything searching the web for maven "filesets" or "assemblies" ... so, would have to look at Tycho code, and make feature request there if we ever wanted that ability (but, seems minor, given we've like not to have a custom plugin at all! :)
(In reply to comment #11) > In regular maven projects, a common practice is to upversion all of the POMs > (like from 4.3.0-SNAPSHOT to 4.3.0), do the release build, and then > upversion all of the POMs to the next version (in our case, 4.4.0-SNAPSHOT) > > As things stand now, we probably expect to upversion our parent poms/repo > poms from 4.3.0-SNAPSHOT to 4.4.0-SNAPSHOT for Luna and 4.3.1-SNAPSHOT for > Kepler SR1. > > Another potential option (I'm less supportive of) would be to continue to > use 4.3.0 for 4.3.x (since it's deployed as a snapshot to repo.eclipse.org) > ... we need the versions distinguished if we need to build 2 branches at the > same time (like 4.3.x and 4.4.0) so we would still have to upversion all of > the references to parent in the poms to 4.4.0-SNAPSHOT. > > To the best of my knowledge, it's not easy to use properties/variables in > parent version/artifactId/groupId. > > > This is another perspective on releases: > http://blog.xebia.com/2012/09/30/continuous-releasing-of-maven-artifacts/ > I'm not really sure how to apply it. Assuming global build configuration, i.e. content of parent pom files, does not change very frequently during release, I recommend avoiding -SNAPSHOT versions for parent poms and use expanded timestamped versions like 4.4.0.20130617-1234, to version parent poms. When global configuration changes, you change parent pom version with new timestamp and update all references to the pom. This way global configuration used by each build will be fully reproducible with relatively low effort. Version update shouldn't be hard to automate too.
(In reply to comment #16) > (In reply to comment #11) > > This is another perspective on releases: > > http://blog.xebia.com/2012/09/30/continuous-releasing-of-maven-artifacts/ > > I'm not really sure how to apply it. > > Assuming global build configuration, i.e. content of parent pom files, does > not change very frequently during release, I recommend avoiding -SNAPSHOT > versions for parent poms and use expanded timestamped versions like > 4.4.0.20130617-1234, to version parent poms. When global configuration > changes, you change parent pom version with new timestamp and update all > references to the pom. This way global configuration used by each build will > be fully reproducible with relatively low effort. Version update shouldn't > be hard to automate too. Sounds good, but not sure its for us, currently. Maybe for maintenance stream, but not sure how it helps reproducibility ... we depend on Git tags for that. Not sure how having a versioned parent POM would help. For main development stream, though, I'd say it changes 1 to 3 times per milestone, and there are roughly 30 other references to it ... so off hand, the hard part, even if partially automated, would rely on getting 5 to 10 other committers to all update their repositories which the patched update "at the same time" (e.g. within an 8 to 12 hour period?) ... and that would be the hardest part. Plus, I'm guessing before long we'll need to use 0.19.0-SNAPSHOT of tycho (and extras) ... so ... if tycho doesn't do it, not sure it helps us much. But, I'm open to hearing how it helps reproducibility ... always one of my favorite topics.
Using our "production build" scripts, I demonstrated we could use tags in repositories.txt and get exactly same build (except for "branding bundles, which are always versioned with "build date"). See bug 410743 comment 12.
Whoops, put comment 18 in wrong bug. I meant to resolve bug 410743.
FYI, I opened bug 410966 to discuss the changes in POM versions and will put on agenda for discussion at Eclipse's Planning/Architecture status meeting for Wednesday.
Who is going to create the branches for each project?
(In reply to comment #21) > Who is going to create the branches for each project? I'd forgotten that's not as easy for everyone as it is for me :) Ideally, if a committer on the project has shell access, then that committer on the project would. For projects where that is not possible, or feasible, I happily volunteer to work with committers to create their persistent branches ... such as arrange ahead of time an "hour long window" or similar to remove the restriction on server bare repository. Good topic for discussion if "e4Build" ID should just do the whole thing ... I'm not sure if git retains a record of "who" created a branch ... or if anyone cares about "committer purity" in this case ... since creating a branch is not exactly IP sensitive. Good topic for Wednesday's status meeting?
(In reply to comment #22) > (In reply to comment #21) > > Who is going to create the branches for each project? > > Good topic for > discussion if "e4Build" ID should just do the whole thing ... -1, at least for "my" repos. I still expect that I only have to branch when I have a real change. > I'm not sure if git retains a record of "who" created a branch ... It's implicitly there: the creator is the owner of the branch file. > or if anyone cares about "committer purity" in this case ... As you might have thought, I do ;-). If a branch was not set up correctly, it allows to know who did it. > since creating a branch is not exactly IP sensitive. If the branch is (intentionally or unintentionally) created from some experimenting commit which did not go through IP review, then it can be an IP issue.
I think we are all done here. I have opened bug 412147 to cover the case of (eventually) creating separate Hudson jobs to do the testing, but don't see any immediate need for that, and might be some easier ways to do it in next major upgrade of Hudson.
