Community
Participate
Working Groups
And perhaps basebuilder should be changed to a product build, see bug 324682
Note there is also bug 350627 about the builder changes that go with this. There are some patches on that bug that were never applied but might still be useful.
The basebuilder will be covered by bug 324682, so changing summary. PW
OK, I've published a version at git clone ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git A much simpler repo than most we've converted. It contains the project in the root, and if we don't like that we should re-convert it (soon :-) I tagged the CVS repo git_migration_starting_point. David can decide when we should move over to the new repo. We can create a patch from this tag to HEAD and apply it if it is too long. PW
(In reply to comment #3) > OK, I've published a version at > > git clone > ssh://userid@git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git > Just curious, I would have thought this would have been just a new project in eclipse.platform.releng/.git Is there a reason to have a whole separate repo? I'm not saying yeah or nay, just wondering what drives that choice?
(In reply to comment #4) > > Just curious, I would have thought this would have been just a new project in > eclipse.platform.releng/.git > > Is there a reason to have a whole separate repo? I'm not saying yeah or nay, > just wondering what drives that choice? That repo is already created. I can't weave 2 repos together properly. To weave them into 1 repo, we would need to go back to CVS for all of the contents, convert them to git, and then use the current eclipse.platform.releng repo to somehow pull all the changes that have happened since the original conversion. I'm open to suggestions, but Kim and I have already failed at this twice :-) PW
current resources on this: http://bugsquash.blogspot.com/2010/03/stitching-git-histories.html http://stackoverflow.com/questions/277029/combining-multiple-git-repositories PW
Just to confirm, when I add this repo, the project it imports ends up named org.eclipse.releng.eclipsebuilder.new (was that new ".new" intended? ... If so, fine, just confirming). Second question, while we'd soon want to use "master" for the 4.2 primary build on eclipse.org, I'm assuming for now, the branch corresponding to what we are currently using, from cvs eclipse builder, is R4_2_primary? Having R_4_2_primary in git checked out and doing a "compare with" the cvs R4_2_primary version seems to show lots of differences. So ... just wondering.
(In reply to comment #7) > Just to confirm, when I add this repo, the project it imports ends up named > > org.eclipse.releng.eclipsebuilder.new No, I wasn't expecting that. I'll go have a look. > Second question, while we'd soon want to use "master" for the 4.2 primary build > on eclipse.org, I'm assuming for now, the branch corresponding to what we are > currently using, from cvs eclipse builder, is R4_2_primary? Having > R_4_2_primary in git checked out and doing a "compare with" the cvs > R4_2_primary version seems to show lots of differences. So ... just wondering. Right, I would expect to use the same branch we were just using, R4_2_primary. PW
(In reply to comment #8) > (In reply to comment #7) Having > > R_4_2_primary in git checked out and doing a "compare with" the cvs > > R4_2_primary version seems to show lots of differences. So ... just wondering. I take that back, the look very similar now ... I must have had wrong projects selected.
(In reply to comment #7) > Just to confirm, when I add this repo, the project it imports ends up named > > org.eclipse.releng.eclipsebuilder.new Apparently that's been in the .project name for a while. I removed it and pushed. PW
I think we can count this as "done", though review (of "design") and cleanup wouldn't hurt. For example, I think we are near ready to remove these files: bootstrap.sh bootstrap42.sh bootstrapHudson.sh bootstrapHudsone4.sh bootstrapHudsongit.sh bootstrapHudsonI.sh bootstrapHudsonicu.sh bootstrapHudsonp2.sh bootstrapTag.sh bootstrapTests.sh Not sure what they are all for, so guess we can leave until positive. But, I created a "scripts" directory, and will use that from now on to put the scripts we are using on build.eclipse.org, mainly masterBuild.sh is the main one to kick things off. Not sure of "platform practices", but assume the version in cvs should be given some "final version used" tag, then the "head version" of that branch nulled out except for a readme file saying that its been migrated to git, incase anyone has it loaded up in their workspace? (it'd be more obvious). Now, for what I'd like review comments on: I worked this into the masterBuild script (via another script, getEclipseBuilder.sh) [I'm guessing more could be converted to Ant, eventually, but ... this works well for now]. Now, the questionable part, I remove the eclipse builder directory used during the build, since it is modified during build (see bug 375780). That's in diretory .../build/supportDir/eclipse.platform.releng.eclipsebuilder I then clone. or if exists already update, a version in 'gitClones' area, namely .../build/supportDir/gitClones/eclipse.platform.releng.eclipsebuilder and use that "fresh" copy to copy back to the directory used during the build. I think this works very well ... but ... I'm not entirely sure what "gitClones" directory was intended for ... so ... am I using as intended? Or offbase?
(In reply to comment #11) > ... though review (of "design") wouldn't hurt. I did have a slight problem ... I was making changes to eclipsebuilder, but they were not getting "pulled in" (literally, during a "pull"). These seemed to work in my little local tests .... so, not sure if something was wrong the first time ... or, if more needs to be done to avoid a "detached head" (if that's the right term?). In either case, I did manually remove the "git repo" from file system, and on the next "clone" it did pick up all the right versions of things. The initial clone looks like: git clone $fullRepoURL cd $repodirectory git config --add user.email "$gitEmail" git config --add user.name "$gitName" Subsequent "pulls" (once local clone exists) look like git checkout $eclipsebuilderBranch git pull So ... seem sane? There could be a dozen other mistakes I'm making (related to directory names or something) but thought I'd ask for a check of these basic git commands.
(In reply to comment #12) > > Subsequent "pulls" (once local clone exists) look like > > git checkout $eclipsebuilderBranch > git pull > > So ... seem sane? There could be a dozen other mistakes I'm making (related to > directory names or something) but thought I'd ask for a check of these basic > git commands. The above looks correct, the checkout switches to $eclipsebuilderBranch (if it's already there its a no-op) and the pull does a fetch origin and a merge origin/$eclipsebuilderBranch Ways it could fail: 1) there's untracked or staged content. Both the checkout and the pull would fail in that case 2) we reset or rebased origin/$eclipsebuilderBranch between pulls. The checkout would work but the pull would fail. In either case there should have been a git error message. PW
(In reply to comment #11) > Not sure of "platform practices", but assume the version in cvs should be given > some "final version used" tag, then the "head version" of that branch nulled > out except for a readme file saying that its been migrated to git, incase > anyone has it loaded up in their workspace? (it'd be more obvious). +1 from me. Deleting HEAD and replacing with a README.txt pointing to the new location is the clearest thing to do.
(In reply to comment #13) > (In reply to comment #12) > > In either case there should have been a git error message. > Well, who looks at those? :) I did now, and I see part of the issue (or, my confusion) is that the new git repo "contains the project in the root" as you say above. I think that means that for the code/scripts we have so far, what used to be named "org.eclipse.releng.eclipsebuilder" is now named "eclipse.platform.releng.eclipsebuilder". At first, I thought "fine", I'll just do a global change of "org.eclipse.releng.eclipsebuilder" to "eclipse.platform.releng.eclipsebuilder" ... but, then, whow, that's sprinkled all over the place! 80% of the cases are probably not even used anymore (lots of old scripts in there, I think) ... but, figured I'd better ask first ... is that the correct direction we want to go? There are probably other approaches, since I'm copying the gitrepo/project to its own directory anyway, I can keep the old name in the directory ... but ... I suspect it will be a perpetual point of confusion? Or, is there some fancy (but easy :) git solution to "rename" the repo/project as its checked out?
FYI, I'm going to be headed down the path of leaving repo and scripts alone, and effectively "rename" copying from repo to the "working area". Seems least change. Let me know if anyone prefers a different approach (or, something to improve in future).
(In reply to comment #15) We can re-run the migration with a slightly different starting point, and we'll have a repo that looks like e.p.releng.maps (with the project in it). > > Or, is there some fancy (but easy :) git solution to "rename" the repo/project > as its checked out? When you clone you can specify the clone-into directory. But yes it provides a different pattern than the others. If we wanted to re-clone the CVS repo, now is the time to do it. We would only have a small set of changes to migrate over to the new repo. PW
I've created a temporary new repo at ssh://id@git.eclipse.org/gitroot/platform/new.eclipse.platform.releng.eclipsebuilder.git I can use format-patch to generates the commits that have been made to the eclipse.platform.releng.eclipsebuilder.git. If I go back to the previous git migration tag and create a new branch I can start applying them, until I hit the set-default-encoding-for-project-to-UTF-8-and-default... commit. Then git am yacks on fixing the line endings. I'm not sure if I can simply re-create the change and then keep going forward. PW
(In reply to comment #18) > I've created a temporary new repo at ssh://id@git.eclipse.org/gitroot/platform/new.eclipse.platform.releng.eclipsebuilder.git OK, I've converted the CVS eclipsebuilder to the above repo, creating a R4_2_primary branch. Then I took our current eclipsebuilder git repo, and used format-patch to replay all of David's commits up until Apr 3, 02:35:28 in a cvs_merge_01 branch (which I've pushed up). Then I merged cvs_merge_01 into R4_2_primary and pushed that up. With the exception of 3 cosmetic differences, our eclipse.platform.releng.eclipsebuilder.git/R4_2_primary and new.eclipse.platform.releng.eclipsebuilder.git/R4_2_primary are identical. David, if you can confirm that the new repo is decent, we can switch out the current one in gitroot and mail out to platform-releng-dev that we've upgrade the repo. PW
(In reply to comment #15) > Or, is there some fancy (but easy :) git solution to "rename" the repo/project > as its checked out? I've given this some thought. If we go with the new archive, we'll be consistent with what we did for e.p.releng.maps.git, in that it will contain the project in question. But we'll still have to go around changing where we get it from, as a clone of the repo will produce <repo>/o.e.releng.eclipsebuilder/... That might be acceptable though. If we stick with our current repo, we can clone it with: git clone -b R4_2_primary \ file:///gitroot/platform/eclipse.platform.releng.eclipsebuilder.git \ org.eclipse.releng.eclipsebuilder or if we just need a copy to work with: git archive --prefix=org.eclipse.releng.eclipsebuilder/ \ --remote=file:///gitroot/platform/eclipse.platform.releng.eclipsebuilder.git \ R4_2_primary | tar xf - So it really just matters what we want to do with this repo? John, do you have an opinion? PW
It seems the new project in the new new repo :) is (still) named org.eclipse.releng.eclipsebuilder.new even though now (at the end) the .project file has been fixed. Given all the issues and complications, I suggest we just stick with /gitroot/platform/eclipse.platform.releng.eclipsebuilder.git and learn (and document) the new pattern of checking out with a different name (simply for compatibility with old scripts). (I'm even having a heck of a time "getting it into my workspace ... since there's so many other "conflicting names" and hard for me to know which (and how) to save/move/remove the "old" ones).
I have confirmed the contents of R4_2_primary matches what's expected except for one attribute change (not sure why/how that didn't work) and a few line endings which are no big deal. I also looked at "history" and seems similar from quick peek. So, my main concern is the new new "org.eclipse.releng.eclipsebuilder" is actually named "org.eclipse.releng.eclipsebuilder.new" as a project.
(In reply to comment #22) > > So, my main concern is the new new "org.eclipse.releng.eclipsebuilder" is > actually named "org.eclipse.releng.eclipsebuilder.new" as a project. I meant to add .... I don't know implications of renaming a project in git, if history is lost? etc. (I think I recall hearing not ... but ... would prefer a git expert confirm). But, again, I'm happy sticking with original new git repo, and learning to "rename on the fly" as needed.
I think we should just clone the CVS repo and if needed do the tweaks in Git. If course it would be good to tag the current state in CVS before cloning.
(In reply to comment #22) > So, my main concern is the new new "org.eclipse.releng.eclipsebuilder" is > actually named "org.eclipse.releng.eclipsebuilder.new" as a project. If I clone the new repo, and then use git Import projects, I correctly get a org.eclipse.releng.eclipsebuilder project. But I thought from comment #21 your recommendation is we stick with the git repo we have, git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git and then take advantage of git clone or git archive to make sure it has the correct o.e.releng.eclipsebuilder directory name when the build needs it. I'm fine with this option as well. PW
(In reply to comment #25) > (In reply to comment #22) > > But I thought from comment #21 your recommendation is we stick with the git > repo we have, > git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git Yes, just didn't know if you were waiting for opinion from John, but, sounds concluded. We'll stick with git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git and use git clone -b R4_2_primary \ file:///gitroot/platform/eclipse.platform.releng.eclipsebuilder.git \ org.eclipse.releng.eclipsebuilder or if we just need a copy to work with: git archive --prefix=org.eclipse.releng.eclipsebuilder/ \ --remote=file:///gitroot/platform/eclipse.platform.releng.eclipsebuilder.git \ R4_2_primary | tar xf - (I haven't tried those yet, but look like they will work fine). I'll leave this bug open a big longer, until I've officially "closed" the cvs versions with readme files.
(In reply to comment #26) > Yes, just didn't know if you were waiting for opinion from John, but, sounds > concluded. We'll stick with > > git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git I've removed the new.eclipse.platform.releng.eclipsebuilder.git repo from git.eclipse.org. There can be only one. PW
I have "nulled out" the cvs versions. I did this only for HEAD and the R4_2_primary branch as I think only ones being looked at recently. My guess is the version we use for 3.8 builds will be a branch (if needed) of the R4_2_primary in git (not a version of the R3_8_primary in cvs). I left a nice long readme.txt file as the only remaining file (in addition to .project file): = = = = = The repository has been migrated to git. ssh://id@git.eclipse.org/gitroot/platform/eclipse.platform.releng.eclipsebuilder.git This migration was done a little different that some, so the files are at the "root" of the repo. This makes the name of the "project" the same as the repo, namely eclipse.platform.releng.eclipsebuilder. (And, confusingly, depending on preference settings, it may still be named "org.eclipse.releng.eclipsebuilder" when checked out into IDE, due to the name in the .project file). Since previous project name was "org.eclipse.releng.basebuilder", and that name is used in many of our build scripts, there are special ways to "check out the git project" but have it "go into to" a directory with old name. See bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=374974 and git docs for details, but in brief, use something similar to git clone -b R4_2_primary \ file:///gitroot/platform/eclipse.platform.releng.eclipsebuilder.git \ org.eclipse.releng.eclipsebuilder or if just need a copy to work with: git archive --prefix=org.eclipse.releng.eclipsebuilder/ \ --remote=file:///gitroot/platform/eclipse.platform.releng.eclipsebuilder.git \ R4_2_primary | tar xf - = = = = =
This didn't start off as blocking move to 4.2 as primary build on build.eclipse.org, but sort of became that way, so marking as blocks 355430.
oh, and, meant to say ... Thanks Paul!
Just to be clear, is this an accurate description of the branches of eclipsebuilder in Git? R4_2_primary -> 4.2 build on eclipse.org R3_8_eclipse -> 3.8 build on eclipse.org master -> Obsolete 3.8 build on IBM build machine R4_HEAD -> Obsolete - I suspect this is the old "4.x long build" on eclipse.org. There have been a couple of commits in here over the past six months but I suspect they were in error. Once the dust settles I think we should end up with something like: master -> 4.2 build on eclipse.org R3_development -> 3.8 build on eclipse.org
(In reply to comment #31) > Just to be clear, is this an accurate description of the branches of > eclipsebuilder in Git? > > R4_2_primary -> 4.2 build on eclipse.org > R3_8_eclipse -> 3.8 build on eclipse.org Yes, I think these were correct. R3_8_eclipse should contain some of the work Kim did to move to eclipse.org for 3.8 before we switched to focusing on 4.2 PW
(In reply to comment #31) > Once the dust settles I think we should end up with something like: > > master -> 4.2 build on eclipse.org > R3_development -> 3.8 build on eclipse.org I definitely agree that what is R4_2_primary should become master on build.eclipse.org. Should we try that "now"? (like, this weekend, I think it may be a slow weekend, due to holidays in some countries?) Eventually, such as by M7 and RC's, we'll want to start using "tagged" versions, for reproducibility, right? (well, actually only needed "for the release" ... but figure we should "practice" with tags before then :) I'm not sure, yet, but suspect that 3.8 could be build with same builder, just a few parameter differences. I know for WTP, we always tried to keep "the builder" the same, and anything that was "stream specific" keep, essentially, in the "maps" project, since it is always stream specific. At times, we would deviate from that pattern for "last minute safety", but it worked well most of the time -- and overall (the reason we did it) was that it was a lot less work, since we didn't have to figure out how to "merge" (cherry pick?) fixes from one branch to the other ... just make the fix once. I don't think there is too many difference between the two builders, at the moment, but hadn't examined in detail. But, if seems like it saves work, would there be objections in principle? Should we open other bug to track "moving to master" and "3.8 builder" ... this bug is, after all, resolved as fixed :)
(In reply to comment #33) > I definitely agree that what is R4_2_primary should become master on > build.eclipse.org. Should we try that "now"? (like, this weekend, I think it > may be a slow weekend, due to holidays in some countries?) It's fine with me. We should also have a tag or branch pointing to the old master so we have a reference point of the last known good IBM internal build. > Eventually, such as by M7 and RC's, we'll want to start using "tagged" > versions, for reproducibility, right? (well, actually only needed "for the > release" ... but figure we should "practice" with tags before then :) Hm, I know the basebuilder contains stable versions of all the bundles used by the build, but I don't know how we managed adoption of new eclipsebuilder in the past. I agree for reproducibility we would need a stable reference point. > I'm not sure, yet, but suspect that 3.8 could be build with same builder, just > a few parameter differences. I know for WTP, we always tried to keep "the > builder" the same, and anything that was "stream specific" keep, essentially, > in the "maps" project, since it is always stream specific. I know we were attempting to move in that direction but currently there are a few artifacts that are different, most notably the product files and p2.inf files. I don't have a feel for how much work this will be to factor out but I think at least in the near term we will need two branches. > Should we open other bug to track "moving to master" and "3.8 builder" ... this bug is, after all, resolved as fixed :) Yes please.
