Community
Participate
Working Groups
While looking into other problems, I often notice (remaining) problems where various variables are "hardcoded" to wrong values. It may be these variables are not used any longer ... or, many are related to tests, so we just haven't hit them yet. But, I thought I'd use this bug to keep rough track of "variable name cleanup", to have at least some documentation of whats chaging.
Doing a search in eclipse builder for "shared/eclipse" I still see many instances where this is "hard coded" in spots, such as in runtests.xml there is <condition property="cvstest.properties" value="/shared/eclipse/buildtests/cvstest.properties"> I'm not positive what cvstest.properties are, or where they should go, they they should not go at the "top" of /shared/elipse/ ... that's asking for interference once several builds/tests are running at same time. Another example is in "helpernew.xml" ... <ant antfile="/shared/eclipse/e4/kimtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/invokeTestsJSON.xml" target="runTests" /> As a first step, I'll change any instance of "/shared/eclipse/" or "/shared/eclipse/e4" to be ${writableBuildRoot} which should be specific to each major build, such as for eclipse 4.2, it will be /shared/eclipse/eclipse4, for 3.8 will be /shared/eclipse/eclipse3, etc. That will be a step in the write direction, though may required some tests to make sure it is "passed in" to all the scripts that need it. And for "eclipsebuilder" locations, I'll change to be the new "eclipsebuilder" location ... which is ${builderDir}, and would have a value (for 4.2) of /shared/eclpse/eclpse4/build/supportDir/org.eclipse.releng.eclipsebuilder
(In reply to comment #1) > there is <condition property="cvstest.properties" > value="/shared/eclipse/buildtests/cvstest.properties"> > > I'm not positive what cvstest.properties are, or where they should go, they > they should not go at the "top" of /shared/elipse/ ... that's asking for > interference once several builds/tests are running at same time. The Platform's CVS JUnit tests require access to a test CVS server where the tests can commit changes. This property file configures the CVS server settings. Should be easy enough to move this property file anywhere... > Another example is in "helpernew.xml" ... > > <ant > antfile="/shared/eclipse/e4/kimtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/invokeTestsJSON.xml" > target="runTests" /> You might already know this, but just to document, this is used to kickstart the JUnit tests running on Hudson. I suspect this script is being checked out by the builder and could be anywhere..
Just to document it, in masterBuild.sh the variable this variable was defined, but seemed to never be used: relengProject=eclipse.platform.releng And just below it was defined (and later used) relengMapsProject=org.eclipse.releng So, I'm assuming 'relengProject' variable was something left over from previous copy/paste/refactoring and will remove it to avoid confusion and "cr*p creep".
another minor case (which drives me crazy) is part of the script that invokes java had -DbuildId=$buildId \ -Dbuildid=$buildId \ i.e. upper and lower case 'i's in the variable. Now's a good time to clean that up, IMHO. There's only a few files that used the lower case version in helper.xml and helpernew.xml, so I'll remove buildid and replace the few cases of use with ${buildId} (upper case i). Note, there are tons of ant substitution phrases that use lower case, @buildid@ and I'm not touching those ... they seem consistent.
I am, finally, going to delete these old "bootstrap" files from the top of eclipsebuilder. bootstrap.sh bootstrap42.sh bootstrapHudson.sh bootstrapHudsone4.sh bootstrapHudsongit.sh bootstrapHudsonI.sh bootstrapHudsonicu.sh bootstrapHudsonp2.sh bootstrapTag.sh bootstrapTests.sh They are no longer used, a little confusing, get "hits" on searches, if anyone needs one to see something "from history" ... they will be there in git, somewhere.
removed WORKSPACE="$buildDir/workspace" export WORKSPACE from masterBuild.sh Seems this used to be part of "bootstrap" and then later used in the jobs to kicks off cronjobs. Seems that is all going to change now.
I removed a LOT of old functions and commented out code from masterBuild.sh, but instead of just deleting it, I put it in a file called "process-publish-build.sh". While "process-publish-build.sh" won't work, to process and publish the build, as-is, we will need something like that once the build is working and then ready to put on "downloads". We may move some of it back to "masterBuild.sh" or may leave it in separate script? But, for now, wanted to get "junk" out of masterBuild.sh.
I'm removing follwing files from eclipseBuilder. My guess is they were used for "cron jobs" and we may need something similar, but these were all based on old stategy of running "bootstrap" so these would no longer work (and, not sure we'd do it exactly same way ... I'd guess we'd call "masterBuild.sh" directly, with appropriate parameters or property files. runIBuilde4 runIBuildTest runIBuildTestp2 runJUnitTests runNBuildTest runNBuildTestgit
I am removing from all build.properties files from eclipsebuilder #tagMaps=true mapsTagTag=v${buildId} The corruption happening in bug 375807 comment 48 might be related. I do have a vague memory of setting "tagMaps" to true in masterBuild.sh once, thinking it was related to autotagging, so that might have "mis-tagged" some files. .... still couldn't explain how it ends up on ~/.cvspass ... but, time to purse the old ways.
(In reply to comment #9) > I am removing from all build.properties files from eclipsebuilder > > #tagMaps=true > mapsTagTag=v${buildId} > There were some similar things in 'base builder' so removed from there too ... only in R4_2_primary, for now.
I'm also removing skipMaps, such as skipMaps=true from build.properties files where I find it. I've commented in another bug where I think this logic is broken (or, at best, very confusing) and for now the operation will be essentially, "if we already have the maps, we can skip getting them again, but if we don't have them, then get them".
FWIW, I've removed $skipPack $skipSourceBuild from masterBuild.sh and "hard coded them to "true" in the scripts such as buildAll.xml, since they won't work (without some effort we don't plan to do) to wanted to avoid the possible "confusion" (from someone like me, in a week :) might forget and try to manipulate.
FYI, finally removed this from "masterBuild.sh" ... not used in current scripts, as far as I could see. arch="x86_64" archProp="-x86_64" archJavaProp="" processor=$( uname -p ) if [ $processor = ppc -o $processor = ppc64 ]; then archProp="-ppc" archJavaProp="-DarchProp=-ppc" arch="ppc" fi
Just to document it, some variables had a segment added called "40builds", such as targetDir=${buildDir}/40builds/targets I've removed that. Seemed extraneous now, partially (or, maybe especially) since I've moved the "different builds" up higher in the hierarchy, so we'll have, at the moment, /shared/eclipse/eclipse4 /shared/eclipse/eclipse4N /shared/eclipse/eclipse3 Also, there are some variables # sdkResults=$buildDir/$buildTag/$buildTag # sdkBuildDirectory=$buildDir/$buildTag Which are not used by the "build scripts". They appear in places in the "old" publishing and/or testing scripts, but hard for me to see how they'd work now ... and ... even if they did, they don't need to be defined in the "build scripts". As long as we pass around $buildDir and $buildTag we can (re) create them when needed. Just documenting small changes.
(In reply to comment #4) > another minor case (which drives me crazy) is part of the script that invokes > java had > > -DbuildId=$buildId \ > -Dbuildid=$buildId \ > > i.e. upper and lower case 'i's in the variable. > > Now's a good time to clean that up, ... Just to document it, removing the lowercase one did have one side effect. The swt zip packages that are created didn't have the actual buildId in them, the had literally '$buildid'. Since I could not find this in our ant scripts/properties anywhere (just doing global searches) my guess is there is some custom ant/java task somewhere that's assuming buildid (lower case). So, I just added back. will need to examine closer later.
(In reply to comment #15) > (In reply to comment #4) Related to buildId ... I've found it confusing that buildLabel and buildId are defined exactly the same. Namely, ${buildType}{$date}-${time} Traditionally, and in parts our eclipse builder code, buildId was {$date}-${time} and buildLabel was ${buildType}{$date}-${time} I'm not sure if this change was made on purpose, or in haste ... and not sure it has much effect ... but might end up effecting tags or qualifiers or something so thought I'd make this note to aide my memory :)
Another oddity, that has caused me a few lost hours today, so I'm going to make consistent It seems in "old" scripts we used a variable named mapVersionTag, to mean the branch or tag of the maps to use. It seems in a few places in new code/scripts we use relengBranch to mean exactly the same thing. Is there a difference? As far as I can tell, not. That no difference is intended? The code mostly works because of the magic of "passing values on the command line" you can always say -DmapVerstionTag=${relengBranch} (effectively translating from one name to the other) But ... when someone new comes along, and makes one little mistake :) Its very difficult to track down what the cause was when the names are inconsistent. So, unless someone tells me "relengBranch" is significant for reasons I can't see, I'm going to change that variable name to match the older "mapVersionTag". I think we all know by now, it means "tag or branch" :) Do let me know if I'm missing something.
This doesn't seem worth a "new bug", but wanted to note it somewhere, that I have been (from the beginning) capturing a "full log" of the whole build be redirecting it, in the initial "kick off" script, with 2>&1 | tee fullmasterBuildOutput.txt but so far have been just leaving that on build machine, so it gets lost during the next build. So, tonight, I've added a few lines of bash code so that just before we are done, to copy that file to the "buildlogs" location. So, we'll have the log as long as we have the build. I could not see any other comprehensive logs created ... hope I'm not missing them ... but I find them valuable. (e.g. that's where you'd find a record of "git command output" and much more. There are often warnings or error message in there that deserve attention, even if its causing no actual problems, such as bug 377844.
I think this bug has served its purpose for this release. Future changes for Juno will be handled in other, specific bugs. Though ... in future, non-endgame time, would be nice to do even more "clean up".
