Community
Participate
Working Groups
Currently the 3.8 build builds most of the bundles. The 4.2 build re-consumes the 3.8 bundles from the 3.8 repo and adds the 4.2 specific bundles and features. This should be reversed so that the 4.2 build creates most of the bundles and runs the full test suites.
Added to the Juno plan. Plan text: The 3.x and 4.x platform builds currently use different build systems and run on different build hardware. We will consolidate these systems to run all builds on eclipse.org build infrastructure. As part of this process the 4.x build will transition to being the primary Eclipse project build system, with 3.x stream builds performed on a reduced schedule for ongoing maintenance work.
Since John updated the maps in the R4_HEAD branch (bug 367888) I've branched the builder (R4_2_Primary) and run test builds. Issues I've encountered and fixed so far -updated version of jsch referenced in source generation to 1.44 -updated maps to point to versions of releng features in Git instead of CVS -source feature generation of SDK should include included features. It only included a few features because it was reconsuming the bundles built in 3.8. I'll revert these changes for tonight's build because they are in the R4_HEAD stream
Paul, I notice the buildDoc.xml for the platform.doc.isv bundle extracts parts of of the 3.x version of the bundle to generate the doc. I assume this is because we were consuming binary bundles from the 3.8 build. Can this be changed as part of making the 4.2 build primary?
The build proceeded to the part where the master feature is compiled. Yay! Will let it run over the weekend and see if the director calls proceed.
(In reply to comment #3) > Paul, I notice the buildDoc.xml for the platform.doc.isv bundle extracts parts > of of the 3.x version of the bundle to generate the doc. Right, we want the 4.2 doc build to be the same as the 3.8 one (just with the 4.2 content branches where applicable) PW
(In reply to comment #3) > Paul, I notice the buildDoc.xml for the platform.doc.isv bundle extracts parts > of of the 3.x version of the bundle to generate the doc. I assume this is > because we were consuming binary bundles from the 3.8 build. Can this be > changed as part of making the 4.2 build primary? We did this because the 4.2 builder was not capable of building javadoc. It needs the complete source to build the javadoc. So instead the 4.2 build just took the 3.8 javadoc unmodified. This problem should just go away when 4.2 build is building everything.
John, The sdk test feature are failing compile because the org.eclipse.core.runtime.compatibility.auth bundle isn't included in any of the features and thus isn't included in the build. I talked to Paul about this and he said that many of the compatability bundles were taken out of the 4.2 build, and just consumed from the 3.8 repos. How should this be handled in the 4.2 build where we are consuming from source?
In addition to the compatibility bundle, the ui.cocoa bundle is missing from the rcp feature and this bundle is required by the equinox osgi starter kit. Not sure if this can be refactored to use new bundles or if these downloads are still required on the equinox page.
We absolutely don't want org.eclipse.core.runtime.compatibility.auth in Eclipse SDK 4.2. It contains ancient encryption code written for Eclipse 1.0 that has long been a source of problems for export control, etc. It was superceded long ago by Equinox Secure Storage. The old API will still work without that fragment, but will store data in plain text with a warning in the log. This is acceptable binary compatibility and everyone should be using the new Equinox API by now.
(In reply to comment #8) > In addition to the compatibility bundle, the ui.cocoa bundle is missing from > the rcp feature and this bundle is required by the equinox osgi starter kit. > Not sure if this can be refactored to use new bundles or if these downloads are > still required on the equinox page. Paul, is org.eclipse.ui.cocoa still relevant in 4.2 or has it been superceded by something else? I suspect the answer is we just want to remove this from the Equinox features but I'll wait for Paul's input.
Regarding comment 9, I've opened bug 368253. There is still a dependency in org.eclipse.core.runtime on the compatability auth bundle and the fact that it is missing is causing a compile error in my test build. This bundle is not explicitly included in the 4.2 build today but since we consume the 3.8 repos, the dependency is resolved.
Had success building the SDKs today. However, it seems like the build id associated with the SDKs has some metadata from a previous build. I'm rerunning the build right now and should have links tomorrow for testing.
Created attachment 209377 [details] no qualifier IU diff I compared the I20120110-2209 test build to our I20120110-2200 build. I've attached the no-qualifier diff as the 4.2 short build auto-tagged at the beginning, making the map files different :-) format: < I20120110-2200 (short build) --- > I20120110-2209 (test build) probably >50% are missing or incorrect source bundles, ecf and p2. Other changes might be just the different cut of the map files: < org.eclipse.equinox.p2.core=2.2.0 --- > org.eclipse.equinox.p2.core=2.1.0 < org.eclipse.equinox.p2.repository=2.1.100 --- > org.eclipse.equinox.p2.repository=2.1.0 different features, maybe, as I have ui.trace in the short build. < org.eclipse.ui.navigator=3.5.101 < org.eclipse.ui.navigator.source=3.5.101 < org.eclipse.ui.trace=1.0.0 < org.eclipse.ui.trace.source=1.0.0 --- > org.eclipse.ui.navigator=3.5.100 > org.eclipse.ui.navigator.source=3.5.100 PW
Thanks Paul! As an aside, I had to hack the build to complete. This is a list of things I noticed and I have to fix -copy examples in doc generation -copy platform.doc.isv from 3.x is probably not needed anymore when generating doc -is ui.cocoa fragment still needed? Some of the equinox build fails because of this -p2.source features and more are missing -platform repos are not provisioned due to missing source features -equinox page can't be generated because of missing ui.cocoa issues which are required for some of the equinox products
Created attachment 209404 [details] Merge the latest 4.2 short build changes into o.e.releng Kim, I've tried to captures in this patch the latest 4.2 build info from I20120110-2200 PW
Thanks Paul! I ran another test build last night. Now, I'm trying to compile the tests and they are failing with builds/I201201121604/org.eclipse.releng.basebuilder/plugins/org.eclipse.pde.build_3.8.0.v20110728/scripts/genericTargets.xml:111: Processing inclusion from feature org.eclipse.sdk.tests: Bundle org.eclipse.update.tests.core_3.3.0.v20111010-0755 failed to resolve.: Missing required plug-in org.eclipse.update.core_0.0.0. Missing required plug-in org.eclipse.help.appserver_0.0.0. These bundles are included in the maps but not in any features any more. They are required by the update.core test bundle org.eclipse.update.tests.core/META-INF/MANIFEST.MF: org.eclipse.update.core, org.eclipse.update.tests.core/META-INF/MANIFEST.MF: org.eclipse.help.appserver, I assume we can stop running the update core tests in the 4.2 stream builds?
(In reply to comment #16) > I assume we can stop running the update core tests in the 4.2 stream builds? Yes. What's the best way to do that? There must be some test feature they need to be removed from?
Created attachment 209557 [details] compile error log from ui.tests.performance from test build
Created attachment 209558 [details] compile error from ui.tests bundle during test build
(In reply to comment #18) > Created attachment 209557 [details] > compile error log from ui.tests.performance from test build Looks like we're not picking up the correct version of this, it should be the 1.1.300 version. PW
(In reply to comment #19) > Created attachment 209558 [details] > compile error from ui.tests bundle during test build Same issue, this looks like the 3.8 version of the plugin as opposed to the one in master. I was hoping the auto-tagging scripts would pick this up ... maybe I need to tag these 2 projects manually? PW
I don't have the autotagging working with this branch of org.eclipse.releng (R4_HEAD) because I don't want to pollute the integration branch of with changes that aren't ready for prime time yet. I'll update the releng maps with the 4.2 versions, thanks Paul :-)
org.eclipse.ui.tests.performance should be tag v20111010-0727 org.eclipse.ui.tests should be v20120106-1348, which I've just pushed. PW
I'll go ahead now and test these scripts to run from cron on build.eclipse.org. It looks like hudson at eclipse is still intermittently unstable.
*** Bug 324687 has been marked as a duplicate of this bug. ***
There are test results a build a ran yesterday for Linux and Windows on Hudson. The Mac tests aren't running due to bug 372866. The map files that were used to run this build are a few days old. These are just the classic SDK tests, upcoming test builds will include the e4 tests as well. Windows https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/JUnit-win2/68/testReport/ DNF ua.tests, ui.tests.rcp p2 tests failures are due to missing properties in test dir, I'll fix this pde build tests failures are expected Linux https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-JUnit-Linux2/ DNF ui.tests.rcp p2 tests failures are due to missing properties in test dir, I'll fix this pde build tests failures are expected Other notes: There's missing javadoc in the build, this is a build issue that will be fixed we we do all the changes that are needed to run the 4.2 build. The build is here http://build.eclipse.org/eclipse/e4/kimtest/build/e4/I20120228-1047/ (Yes, the php on the build page is wonky, I have to fix that)
The webmasters fixed an issue with the Mac machine yesterday and here are the mac results for the same build https://hudson.eclipse.org/hudson/view/Eclipse%20and%20Equinox/job/eclipse-JUnit-mac2/lastCompletedBuild/testReport/ Same issues as well as p2.ui, p2 and ui.tests.rcp DNF
Just documenting the current issues here The builder to build the 4.2 primary build is the R4_2_primary branch of org.eclipse.releng.basebuilder and org.eclipse.releng.eclipsebuilder The branch of the map files with the bundles to build are R4_HEAD. There are a few weeks out of date. This is just for testing. e4Build@build:/shared/eclipse/e4> pwd /shared/eclipse/e4 e4Build@build:/shared/eclipse/e4> ls masterBuildKim.sh masterBuildKim.sh Is the script that invokes the build. It needs to be refactored a bit. Also, the bits that tag the repos have been commented out because I don't want to pollute the the 4.2 maps until we have the other issues worked out. In our regular 3.8 stream builds we have scripts in the bootstrap script that update the tags in the map files repo. The eclipse.platform.releng.maps\org.eclipse.releng\tagging\repositories.txt file lists the appropriate branches to look for each repo. These scripts update the maps with a timestamp in the tags field of the appropriate project. This is disabled for now for testing purposes. Anyways, the issues and notes 1) The current 4.2 stream org.eclipse.platform.doc.isv bundle copies over some content from the 3.x stream of this same bundle. This needs to be deleted once we move to 4.2 build primary 2) update.core has been added added to sdk.tests feature because some of the tests still depend on it. 3) update.tests.core has been removed for org.eclipse.sdk.tests feature because these tests aren't relevant anymore 4) The 4.2 versions sdk and platform features have parts commented in their build.properties that are commented out. These need to be uncommented and released when we move to the 4.2 build primary so the appropriate source bundles are produced. I've temporarily uncommented them and tagged them for the R4_HEAD version of org.eclipse.releng. 5) The e4 build part doesn't run yet, I just build the e4 bundles needed for the platform. 6) Need to add the 4.2 stream tests to the sdk.tests feature and run them, right now it just includes the 3.8 stream tests. 7) The test results page generation also needs to be updated to include all the tests. 8)I use a hudson token to invoke the tests via JSON. This needs to be stored somewhere secure. Right now, I just add it to the command.txt when hacking the build. 9) The build isn't copied to the correct location, signed or mirrored into the correct repo because it's just a test build. http://wiki.eclipse.org/Platform-releng-basics#Restarting_the_build
(In reply to comment #26) > The build is here > http://build.eclipse.org/eclipse/e4/kimtest/build/e4/I20120228-1047/ This was very helpful, thanks! For the Object Teams build it would be good to know if another build of this kind is planned for M6. I guess the official M6 build will still be 3.8-first, right?
>>I guess the official M6 build will still be 3.8-first, right? Yes Some other issues I thought of last night We run unix2dos on runtests.bat in the customTargets.xml for building the sdk.tests. And dos2unix on runtests. To ensure the test scripts have the correct line delimiters. The other thing to mention is that there are a number of bundles that are common to the 3.8 stream and 4.2 stream but consumed from different branches. They are platform.doc.isv platform.doc.user org.eclipse.ui org.eclipse.ui.workbench features org.eclipse.rcp org.eclipse.platform org.eclipse.sdk As I mentioned before, the platform and sdk features need to have some things uncommented in their build.properties so their source features properly generate once we build all bundles from source in the 4.2 build as opposed to consuming the 3.8 ones.
We have a 4.2 short build scheduled at 22:00 EDT tonight. Tomorrow maybe we can start running the 4.2 primary build. So what I'd like to do to move forward (comments welcome): I'd suggest creating a R42_build_test branch on all of the repos before we start, and update tagging/repositories.txt to use that. (In reply to comment #28) > > In our regular 3.8 stream builds we have scripts in the bootstrap script that > update the tags in the map files repo. The > eclipse.platform.releng.maps\org.eclipse.releng\tagging\repositories.txt file > lists the appropriate branches to look for each repo. If we update the repositories.txt file, we can go ahead and tag and push to our hearts content. > > Anyways, the issues and notes > 1) The current 4.2 stream org.eclipse.platform.doc.isv bundle copies over some > content from the 3.x stream of this same bundle. This needs to be deleted once > we move to 4.2 build primary > 2) update.core has been added added to sdk.tests feature because some of the > tests still depend on it. > 3) update.tests.core has been removed for org.eclipse.sdk.tests feature because > these tests aren't relevant anymore > 4) The 4.2 versions sdk and platform features have parts commented in their > build.properties that are commented out. These need to be uncommented and > released when we move to the 4.2 build primary so the appropriate source > bundles are produced. I've temporarily uncommented them and tagged them for > the R4_HEAD version of org.eclipse.releng. John, can we make the source changes #1-#4 in the R42_build_test branches on the appropriate repos. Get the test as close as possible. > 5) The e4 build part doesn't run yet, I just build the e4 bundles needed for > the platform. After a successful test, we should update the e4 masterBuild.sh to run only the e4 build against the test repo. > 6) Need to add the 4.2 stream tests to the sdk.tests feature and run them, > right now it just includes the 3.8 stream tests. > 7) The test results page generation also needs to be updated to include all the > tests. Then we'll start working through these. > 9) The build isn't copied to the correct location, signed or mirrored into the > correct repo because it's just a test build. Once we have close-enough test build, we should have a close-enough signed and published test build that we just agree to delete afterwards. The autotagging should be fine with the different streams for the workbench in 3.8/4.2 and with the other repos, that only have one branch for both builds. PW
I've pushed a change to repositories.txt to make sure it's pointing to all the correct branches. The auto-tagging should correctly update jdt and pde map files based on their integration branches, even though the tags in question have already been pushed. PW
(In reply to comment #31) > John, can we make the source changes #1-#4 in the R42_build_test branches on > the appropriate repos. Get the test as close as possible. I have fixed up the build.xml for org.eclipse.platform.doc.isv so it is generating javadoc rather than slurping it from 3.8 build. From our earlier call with Kim it sounds like she has already done the update.core and update.test.core changes. Just to clear up my confusion what branch are we using in the releng repo for 3.8 and 4.2 builds? http://git.eclipse.org/c/platform/eclipse.platform.releng.git/
(In reply to comment #33) > Just to clear up my confusion what branch are we using in the releng repo for > 3.8 and 4.2 builds? > > http://git.eclipse.org/c/platform/eclipse.platform.releng.git/ I assumed going forward it was R4_HEAD. When I updated the repositories.txt file in releng.maps/R4_HEAD, I see: ssh://kmoir@git.eclipse.org/gitroot/platform/eclipse.platform.common.git R4_HEAD ssh://kmoir@git.eclipse.org/gitroot/platform/eclipse.platform.git R4_HEAD ssh://kmoir@git.eclipse.org/gitroot/platform/eclipse.platform.releng.git R4_HEAD everything else comes from master (or integration). Switching them around so master == 4.2 I thought could wait until later. PW
I updated the platform and sdk features to so the build.properties specify the source features to be generated. (item 4 from comment 28). Update.tests.core has been removed from the sdk.tests feature (item 3 from comment 28). Update core has been added to the sdk.tests feature (item 2 from conment 28). Have the R42_build_test branch been created on all repos? Is that still the plan?
(In reply to comment #35) > > Have the R42_build_test branch been created on all repos? Is that still the > plan? No, McQ was fine with just flipping the switch. I've updated repositories.txt in R4_HEAD to use the correct branches for auto-tagging for the rest of the repos. We should just go ahead, including auto-tagging in the test so that the map files get correctly updated. PW
Also forgot to mention that I updated the 4.2 maps to consume the new Ant 1.8.3, ECF, jetty and Orbit as is already consumed in the 3.8 build.
I've tried to run a modified version of masterBuild.sh running on my local machine. and eventually hit FetchFromCVS: password: and a long stretch of ... GitCloneRepoToLocalRepo: GitCheckoutTagInLocalRepo: GitFetchElementFromLocalRepo: GitUpdateLocalRepo: GitCheckSkipClone: GitCloneRepoToLocalRepo: GitCheckoutTagInLocalRepo: GitFetchElementFromLocalRepo: and it is not my password its asking for :) Hope I didn't lock you out Kim :\ From quick peek at /org.eclipse.releng.eclipsebuilder R4_2_primary I can still see "kmoir" in there (as well as ottawa cvs?) I certainly know "how to fix" after Friday ... but, thought I'd ask if I'm missing something obvious. The masterBuild.sh I started with, I think, was masterBuildKim.sh (and changed any mention of pwebster id to my own id (or e4Build) ... just to make sure I didn't do something under his id (and lock him out :) So ... anything obvious? quick suggestions? After Friday, I'll "hack away" at the code in org.eclipse.releng.eclipsebuilder R4_2_primary but didn't want to mess with anything right now, in case Kim (or someone) was still doing something with it Friday. FWIW, I have tried overriding cvsUser and cvsRepo on the java command (which should handle the ant script ... but ... not sure what bash scripts are executing. Any debugging tips you'd use (there than echo's, -d, etc.) Thanks,
(In reply to comment #38) > I've tried to run a modified version of masterBuild.sh running on my local > machine. and eventually hit > > FetchFromCVS: > password: > I did just hit enter here a few times (instead of ctrl-c, and did get a little more info ... FetchFromCVS: [CVS - org.eclipse.equinox/security/bundles/org.eclipse.equinox.security.win32.x86_64] Permission denied, please try again. [CVS - org.eclipse.equinox/security/bundles/org.eclipse.equinox.security.win32.x86_64] Received disconnect from 206.191.52.50: 2: Too many authentication failures for kmoir [CVS - org.eclipse.equinox/security/bundles/org.eclipse.equinox.security.win32.x86_64] cvs [export aborted]: end of file from server (consult above messages if any) Maybe that will mean more to you than it does to me. (Again, sorry Kim if this "locks you out").
(In reply to comment #39) > FetchFromCVS: > [CVS - > org.eclipse.equinox/security/bundles/org.eclipse.equinox.security.win32.x86_64] > Permission denied, please try again. > [CVS - > org.eclipse.equinox/security/bundles/org.eclipse.equinox.security.win32.x86_64] > Received disconnect from 206.191.52.50: 2: Too many authentication > failures for kmoir > [CVS - > org.eclipse.equinox/security/bundles/org.eclipse.equinox.security.win32.x86_64] > cvs [export aborted]: end of file from server (consult above messages if any) According to our map files this fragment should come from Git. kmoir really shouldn't be appearing anywhere in our scripts at this point. While looking through the scripts, I noticed the "cvsuser" variable in several places. All of the references to this are in customTargets.xml files, where it does this: <replace dir="${buildDirectory}/maps/org.eclipse.releng/maps" value="${cvsuser}" token=":pserver:anonymous" /> I think all of these can be removed (and all traces of the "cvsuser" property too) because there is no such variable in our map files.
David, how are you invoking the build on your local machine? I'm currently running masterBuild.sh on build.eclipse.org as e4Build and it's working. If you set hudson to true it should check out the builder as :pserver:anonymous
(In reply to comment #41) > David, how are you invoking the build on your local machine? > > I'm currently running masterBuild.sh on build.eclipse.org as e4Build and it's > working. > > If you set hudson to true it should check out the builder as :pserver:anonymous I am "running as" my local ID, and changed some of the id's hard coded in the script to my committer ID ... maybe that should be e4Build instead (but, in the example I started with, it had pwebster, ... I know, for tagging, currently) ... but you and John have given good hints as to what might be "going wrong". Thanks!
(In reply to comment #42) > I am "running as" my local ID, and changed some of the id's hard coded in the > script to my committer ID ... maybe that should be e4Build instead (but, in the > example I started with, it had pwebster, ... I know, for tagging, currently) > ... but you and John have given good hints as to what might be "going wrong". > Thanks! I'm wondering why you are working on getting the build running on your home machine when Kim said it already works on build.eclipse.org. Running test builds at eclipse.org would make it easier for us all to pitch in and diagnose any problems. From painful experience, getting the build running on your machine does not imply it will work on build.eclipse.org. We can turn off email notifications if we want to run test builds without disrupting people.
(In reply to comment #43) > (In reply to comment #42) > I'm wondering why you are working on getting the build running on your home > machine when Kim said it already works on build.eclipse.org. Running test > builds at eclipse.org would make it easier for us all to pitch in and diagnose > any problems. From painful experience, getting the build running on your > machine does not imply it will work on build.eclipse.org. We can turn off email > notifications if we want to run test builds without disrupting people. Simply to increase my learning. i.e., I would not say it already works on build.eclipse.org. If it was I wouldn't worry too much about it, but since its not, I'll need to understand it, and quickly. Plus, until now, Kim has also been doing test builds on build.eclipse.org ... so, didn't want to interfere with that ... now that will no longer be being happening, I will being doing both. > From painful experience ... Tell me about it. :)
So I have been trying to run a build from maps today (R4_HEAD) using the script here e4Build@build:/shared/eclipse/e4/masterBuildKim.sh. Here is the error. Not sure why this is happening. It might be a good idea to just start from scratch and merge the maps from org.eclipse.releng master into R4_HEAD and then add the e4 ones. Not sure what plug-in null_0.0.0 means, I have seen that before but don't recall how I resolved this issue. It has not been working. Here is the error message. I have to [eclipse.buildScript] Bundle org.eclipse.jetty.server: [eclipse.buildScript] Unsatisfied import package javax.servlet_2.6.0. [eclipse.buildScript] Unsatisfied import package javax.servlet.descriptor_2.6.0. [eclipse.buildScript] Unsatisfied import package javax.servlet.http_2.6.0. [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.continuation_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.http_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.http.gzip_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.io_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.io.bio_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.io.nio_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.jmx_8.0.0. [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util.component_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util.log_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util.resource_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util.ssl_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util.statistic_[8.1.0,9.0.0). [eclipse.buildScript] Unsatisfied import package org.eclipse.jetty.util.thread_[8.1.0,9.0.0). BUILD FAILED /shared/eclipse/e4/kimtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/buildAll.xml:59: The following error occurred while executing this line: /shared/eclipse/e4/kimtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/buildAll.xml:219: The following error occurred while executing this line: /opt/public/eclipse/e4/kimtest/build/e4/org.eclipse.releng.basebuilder_R4_2_primary/plugins/org.eclipse.pde.build_3.8.0.v20120119-1950/scripts/build.xml:91: The following error occurred while executing this line: /shared/eclipse/e4/kimtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/eclipse/buildConfigs/master/customTargets.xml:18: The following error occurred while executing this line: /shared/eclipse/e4/kimtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/eclipse/buildConfigs/master/allElements.xml:16: The following error occurred while executing this line: /opt/public/eclipse/e4/kimtest/build/e4/org.eclipse.releng.basebuilder_R4_2_primary/plugins/org.eclipse.pde.build_3.8.0.v20120119-1950/scripts/genericTargets.xml:111: Processing inclusion from feature master-jetty: Bundle org.eclipse.equinox.http.servlet_1.1.300.v20111202-1521 failed to resolve.: Unsatisfied import package javax.servlet_[2.3.0,3.1.0). Unsatisfied import package javax.servlet.annotation_2.6.0. Unsatisfied import package javax.servlet.descriptor_2.6.0. Unsatisfied import package javax.servlet.http_[2.3.0,3.1.0). Host plug-in null_0.0.0 has not been found.
(In reply to comment #45) > So I have been trying to run a build from maps today (R4_HEAD) using the script > here e4Build@build:/shared/eclipse/e4/masterBuildKim.sh. Here is the error. > Not sure why this is happening. It might be a good idea to just start from > scratch and merge the maps from org.eclipse.releng master into R4_HEAD and then > add the e4 ones. Not sure what plug-in null_0.0.0 means, I have seen that > before but don't recall how I resolved this issue. > I noticed failures related to the fact that the Ant 1.8.3 bundle was no longer on "build.eclipse.org" ... so, I (tried) to update the R4_HEAD maps to correct that ... maybe that'll magically fix this other issue? But, I was getting a clear message "could not find ...build.eclipse.org...ant_1.8.3" ... yes, on my local machine :) Other notes, I've put a script similar to Kim's called masterBuildDW2.sh in /shared/eclipse/e4 and will "create results" in /shared/eclipse/e4/dwtest (instead of kimtest). Its running now. (yes, under e4Build, with paul's committer id :) Couple of minor things I "fixed": Some "test results" were going into the home folder of e4Build, making that home directory huge (like 80G) ... these normally should be pretty small couple hundred meg at most) ... so ... fixed the script not to use e4Build's home directory. Couple of minor variables and other minor changes. But ... while writing this, I just noticed my script (based on an earlier version's of Kim's (I'm assuming?) had one spot that had mapVersionTag=HEAD instead of what's currently in hers: mapVersionTag=R4_HEAD So, that might have been accounting for the Ant 1.8.3 issue? I'll correct that, and maybe make a few more runs ... and then will need to "break" to get ready and go to EclipseCon.
another note: I'll try to always capture my output in masterBuildDW2out.txt from running ./masterBuildDW2.sh If anyone ever wants to browse the results, for fun. :) Some other notes about Kim's "master" script that _might_ account for some odd problems ... in at least one of the cmd=" ... statements, there were a few places near the end that did not have "continuation" characters, and, maybe worse, some continuation characters followed by a space, before the EOL, which bash doesn't interpret as a continuation characters. For example, cmd=" .... -DCDC-1.0/Foundation-1.0=$bootclasspath_foundation \ -DCDC-1.1/Foundation-1.1=$bootclasspath_foundation11 \ -DOSGi/Minimum-1.2=$OSGiMinimum \ -DJavaSE-1.6=$bootclasspath_16 \ -DlogExtension=.xml \ $javadoc \ $sign \ $repoCache \ -DgenerateFeatureVersionSuffix=true \ -Djava15home=$javaHome -DpostingDirectory=$postingDirectory -noTag" I have no idea of the impact of this, maybe none, but thought I'd mention it in case it leads anyone to an "ah ha" moment. And, I'm assuming that final "-noTag" should have been -DnoTag=true" ? ... I was getting a msg that said "-noTag is an unrecognized command". Again ... could all me minor things ... just making notes as I go. I've restarted another run with mapVersionTag=R4_HEAD
(In reply to comment #45) > ... It might be a good idea to just start from > scratch and merge the maps from org.eclipse.releng master into R4_HEAD and then > add the e4 ones. I'm at this same point now. The final, literal "build failed" is: /opt/public/eclipse/e4/dwtest/build/e4/org.eclipse.releng.basebuilder_R4_2_primary/plugins/org.eclipse.pde.build_3.8.0.v20120119-1950/scripts/genericTargets.xml:111: Processing inclusion from feature org.eclipse.e4.rcp: Bundle org.eclipse.e4.ui.workbench.swt_0.10.0.v20120317-0001 failed to resolve.: Missing required plug-in org.eclipse.e4.ui.workbench_0.10.0 Which seems odd on several levels, all by itself, but was proceeded by an number of the missing host null_000 bundle: generateScript: [eclipse.buildScript] Some inter-plug-in dependencies have not been satisfied. [eclipse.buildScript] Bundle org.eclipse.ui.console: [eclipse.buildScript] Missing required plug-in org.eclipse.ui_[3.5.0,4.0.0). [eclipse.buildScript] Missing required plug-in org.eclipse.ui.workbench.texteditor_[3.5.0,4.0.0). [eclipse.buildScript] Bundle org.eclipse.help.ui: [eclipse.buildScript] Missing required plug-in org.eclipse.ui_[3.6.0,4.0.0). [eclipse.buildScript] Bundle org.eclipse.ui.views.properties.tabbed: [eclipse.buildScript] Missing required plug-in org.eclipse.ui.views_[3.2.0,4.0.0). [eclipse.buildScript] Missing required plug-in org.eclipse.ui_[3.3.0,4.0.0). [eclipse.buildScript] Host plug-in null_0.0.0 has not been found. [eclipse.buildScript] Bundle org.eclipse.equinox.p2.ui: [eclipse.buildScript] Missing required plug-in org.eclipse.ui_3.6.0. [eclipse.buildScript] Missing required plug-in org.eclipse.equinox.security.ui_[1.0.0,2.0.0). [eclipse.buildScript] Host plug-in null_0.0.0 has not been found.
Some other changes I experimented with ... which I don't think contribute to current failure .. but, just in case ... For a while I was getting failures during the "presetup" step, where a local mirror of EMF pre-reqs was trying to be made, and then a "runnable" made from that ... and the runnable task was failing, but seemed to be trying to use an older version of EMF, not "the latest" in the repo as I would have expected. I can imagine a couple of reasons why this was being done, but do not think its central to "doing the main build", especially now that we are on build.eclipse.org, so removed it for now. This also required removing in the prefetch step one of those <replace cvsuser tasks, since that local repo no longer exists. Last "experiment", I commented out the "buildSourceDrops" property in buildEclipseSourceDrops task. We want to do that anyway, see bug 374428, and it seemed to take a long time doing the "source build" package ... and seemed to clutter the whole check-out tree and processing ... so, thought it was a good time to simplify and remove things not strictly needed to "build on eclipse.org".
(In reply to comment #48) > (In reply to comment #45) > > ... It might be a good idea to just start from > > scratch and merge the maps from org.eclipse.releng master into R4_HEAD and then > > add the e4 ones. > > I'm at this same point now. I meant to mention, I actually tried to merge the master and R4_HEAD branches but a) git gives me fits :) and b) not all the changes or differences "made sense" to me .... some things I wouldn't have expected seemed more recent in R4_HEAD than 'master' ... and other things older ... so, figured that'd be a good "team" exercise.
(In reply to comment #48) > [eclipse.buildScript] Bundle org.eclipse.equinox.p2.ui: > [eclipse.buildScript] Missing required plug-in org.eclipse.ui_3.6.0. > [eclipse.buildScript] Missing required plug-in > org.eclipse.equinox.security.ui_[1.0.0,2.0.0). > [eclipse.buildScript] Host plug-in null_0.0.0 has not been found. This message "Host plug-in null_0.0.0 has not been found" is a bug in PDE/Build BuildTimeSite#getResolutionFailureMessage where GenericSpecification constaints are not handled. Starting with bug 368829, Bundle required execution environment constraints are now represented using GenericSpecification. I would suggest checking that the appropriate execution environment bootlasspath properties (eg CDC-1.1/Foundation-1.1, J2SE-1.4, J2SE-1.5, etc) are set properly for the build.
(In reply to comment #51) > (In reply to comment #48) > ... > Starting with bug 368829, Bundle required execution environment constraints are > now represented using GenericSpecification. > > I would suggest checking that the appropriate execution environment > bootlasspath properties (eg CDC-1.1/Foundation-1.1, J2SE-1.4, J2SE-1.5, etc) > are set properly for the build. Looks like a promising hint, thanks Andrew! The "real" JREs look right and do exist, from a quick glance; -DJ2SE-1.5=/shared/common/jdk-1.5.0_16/jre/lib/rt.jar:/shared/common/jdk-1.5.0_16/jre/lib/jsse.jar:/shared/common/jdk-1.5.0_16/jre/lib/jce.jar -DJ2SE-1.4=/shared/common/j2sdk1.4.2_19/jre/lib/rt.jar:/shared/common/j2sdk1.4.2_19/jre/lib/jsse.jar:/shared/common/j2sdk1.4.2_19/jre/lib/jce.jar -DJavaSE-1.6=/shared/common/jdk1.6.0_27.x86_64/jre/lib/rt.jar:/shared/common/jdk1.6.0_27.x86_64/jre/lib/jsse.jar:/shared/common/jdk1.6.0_27.x86_64/jre/lib/jce.jar BUT, the "foundation" level settings seem a bit off, not sure if there was some other, historical reason? Or typos? But, they end up, on the command line, being -DCDC-1.0/Foundation-1.0=/shared/common/org.eclipse.sdk-feature/libs/ee.foundation-1.0.jar -DCDC-1.1/Foundation-1.1=/shared/common/org.eclipse.sdk-feature/libs/ee.foundation.jar -DOSGi/Minimum-1.2=/shared/common/org.eclipse.sdk-feature/libs/ee.foundation.jar Seems at least that last one should be -DOSGi/Minimum-1.2=/shared/common/org.eclipse.sdk-feature/libs/ee.minimum-1.2.0.jar And, could be a typo in readme, but according to the readme in /shared/common/org.eclipse.sdk-feature/libs The Foundation-1.0 level is still prefixed with CDC-1.1 though, could be a typo in the readme file? But, wondering if that first one should be -DCDC-1.1/Foundation-1.0=/shared/common/org.eclipse.sdk-feature/libs/ee.foundation-1.0.jar (same 1.0 jar, but prefixed with "cdc-1.1" instead of "cdc-1.0" ... I'm sure there's some way I could look that up ... if no one knows off the top of their head). But ... will at least make the change for -DOSGi/Minimum-1.2 and see if results any different.
(In reply to comment #52) > (In reply to comment #51) > > (In reply to comment #48) > And, could be a typo in readme, but according to the readme in > /shared/common/org.eclipse.sdk-feature/libs > The Foundation-1.0 level is still prefixed with CDC-1.1 > though, could be a typo in the readme file? > But, wondering if that first one should be > > -DCDC-1.1/Foundation-1.0=/shared/common/org.eclipse.sdk-feature/libs/ee.foundation-1.0.jar > I think typo in readme, from http://en.wikipedia.org/wiki/OSGi seems it should be CDC-1.0/Foundation-1.0 ... that is, correct as is. But, seeing that whole list in the wiki, makes me wonder if we need more? CDC-1.0/Foundation-1.0 CDC-1.1/Foundation-1.1 OSGi/Minimum-1.0 OSGi/Minimum-1.1 JRE-1.1 From J2SE-1.2 up to J2SE-1.6 I assume this would have been known from builds running on previous hardware, so will assume not ... but ... do let me know if anyone thinks we need a a JRE1.1, or 1.3, etc.
(In reply to comment #53) > But, seeing that whole list in the wiki, makes me wonder if we need more? > CDC-1.0/Foundation-1.0 > CDC-1.1/Foundation-1.1 > OSGi/Minimum-1.0 > OSGi/Minimum-1.1 > JRE-1.1 > From J2SE-1.2 up to J2SE-1.6 > > I assume this would have been known from builds running on previous hardware, > so will assume not ... but ... do let me know if anyone thinks we need a a > JRE1.1, or 1.3, etc. You only need to define properties that are referenced from the bundles you are building. I expect this is 1.4, 1.5, 1.6 for most bundles. Then CDC-1.1/Foundation-1.1 and likely OSGi/Minimum-1.2. In general, these properties are mostly to ensure compile time accuracy. For them to cause resolution problems would probably require lower level environments being defined but not higher levels.
Got exact same results after changing to use -DOSGi/Minimum-1.2=/shared/common/org.eclipse.sdk-feature/libs/ee.minimum-1.2.0.jar So ... back to trial and error for me :) (e.g. going to revert the "sourceBuild" change I made ... might be unintended side effects?) ... but, suspect big part of the problem is R4_HEAD maps being "out of date"?
With Paul's help ... we had our own two person BOF :) ... got farther. So, just to document the issue: We looked at merging the maps, and Paul could see (and explain) the parts that were "out of date" but other things were as they should be, and the out of date parts should not cause complete failure ... just build something slightly out of date. The "generate scripts" step was failing so badly it turns out, because emf (common/core) was not in the "runnable target" needed by PDE. While they are listed in some of the map files, that only works if they are "included" ... which Eclipse SDK does not want to do (since tightly couples to exactly one version). As noted earlier, I'd commented that out since it was failing (and I had forgotten the maps weren't enough since not "included") ... so with Paul's help we just worked backwards and discovered that was failing for the simple reason of not passing in -DbuildDirectory=${buildDirectory} on the very initial java invocation that starts the build. sigh So ... it gets much further, I think it is build/compiling everything ... then gets to the part of preparing products and packages (I think). It failed with a message about not being able to copy a p2.inf file to some location while trying to "configure" the rcp product. In short, I think, it was failing here: <!-- get the generate p2.inf that got generated with the above scripts --> <move file="${basedir}/temp/features/org.eclipse.pde.build.container.feature/product/p2.inf" tofile="${basedir}/p2.inf" overwrite="true" /> And, appears the "generated with above scripts" were not working quite right, so I added some echos to test the exact values involved. I started a new job, fresh (removing dwTest direcotory) in case there's some issues with "first time run" vs. "second time run". If the same thing happens again, I may just try to skip the rcp "product build" since I don't think it is an essential piece to "get building" right away on build.eclipse.org, but doubt that'd improve things much. Thanks Paul.
After 6 hours of debugging, discovered that specifying buildDirectory on command line was the wrong fix. The reason is that there are parts of eclipseBuilder that depend on (re)using that variable, with its own special value, such as direct all to generateScripts in generic targets ... hence, putting it on command line locks in one value for the whole run, instead of different parts being able to have their own (scoped) values. Bummer. The "real fix" was me understanding and "fixing" (some of my own damage) to how sensitive the build it to being in the right "starting directory" (and it "changes directories" several times, which in turn changes the ${basedir} for those ant scripts. At any rate, once all that was fixed, the build got pretty far. 53 binaries, 78 features, 757 plugins ... sound about right? This run bailed out in "run.xml" in /equinox/buildConfigs/equinox.prov/run.xml, due to a simple ant error. There is a key part of that ant script that looks like this: <target name="build" depends="init"> <parallel failonany="true" threadCount='4'> <!-- <antcall target="provision.agent" /> <antcall target="provision.starterkit" /> --> <!-- <antcall target="provision.pdeproduct" /> --> <antcall target="provision.sdk" /> <antcall target="provision.platform" /> <antcall target="provision.platformsdk"/> </parallel> For that last one, there really is no provision.platformsdk in this version of the run.xml, so immediate ant error. Not sure why ... seems to be in other (head) version of eclipse builder ... this problem doesn't even show up as an error in IDE, I guess since the ant file is not named "build.xml". :( So .. I'll try commenting out that target call ... and see what breaks next. We can "restore" it later, if its needed.
(In reply to comment #57) > <antcall target="provision.platformsdk"/> commenting this out is the correct thing to do. We had agreed to remove it completely from the build (and this spot must have been missed). PW
Note: running the build twice, in a row, without cleaning/clearing directory, results in a relatively quick failure: /shared/eclipse/e4/dwtest/build/e4/src/assemble.master.p2.xml:25: The following error occurred while executing this line: /shared/eclipse/e4/dwtest/build/e4/src/features/org.eclipse.sdk.examples.source/build.xml:188: Property licenseURL in /shared/eclipse/e4/dwtest/build/e4/src/features/org.eclipse.sdk.examples.source/../org.eclipse.license/feature.properties conflicts with property in /shared/eclipse/e4/dwtest/build/e4/src/features/org.eclipse.sdk.examples.source/feature.temp.folder/features/org.eclipse.sdk.examples.source_3.5.0.v20110912/feature.properties I'll try again with clean directory just making note of observations ... things to solve eventually (well, I'm assuming the build would clean up what it needed to ... and be able to run twice (or more) ... but, not sure of what intent/design is yet.
See bug 374974, I'm taking a crack at this now. PW
(In reply to comment #60) > See bug 374974, I'm taking a crack at this now. With so many other changes going on, why it this a priority now? Typically the fewer variables involved the better ... so, just curious and trying to learn.
(In reply to comment #61) > (In reply to comment #60) > > See bug 374974, I'm taking a crack at this now. > > With so many other changes going on, why it this a priority now? Typically the > fewer variables involved the better ... so, just curious and trying to learn. Partly because I have time now, partly because git is much better at handling dealing with the 2 streams we have to (3.8 and 4.2 primary), especially at keeping some parts of them in sync. Also because it's the last real hold-out w.r.t. CVS (at least in the main Eclipse Project). Working in both git and CVS involves more switching contexts (it's been annoying me over the last couple of days). All in git is, well, simpler. That being said, if you really don't want to do it now it doesn't have to be now (I agree with that the more you change the more variables you introduce). We're pretty confident in our git migration strategy though. I want rid of the CVS stuff, and I had set aside tonight to hack at build stuff anyway. But I'll leave the final call up to you. PW
two steps forward, two steps back ... After commenting out platformsdk and cleaning "dwtest" directory, I did get an SDK build ... seemed to compile, produce the right distributions, etc. That's one step forward. But, there were some compile errors that were a little surprising and Paul thought it'd be a good idea to update maps in R4_HEAD to make sure they were current and maybe the compile errors would be fixed. So, he and I sat down and "fixed" a standalone "git-release.sh" script that just updated the maps in R4_HEAD based on what was just built (the "autotagging" is currently turned off, but the fixed "git-release.sh" is what does the autotagging. It required a number of changes and even had to get Denis involved who just happened to wander by :) and in the end, appeared to work, from looking at web: http://git.eclipse.org/c/platform/eclipse.platform.releng.maps.git/commit/?h=R4_HEAD Another step forward. But, when retrying the build, it fails pretty much immediately, saying, in excerpt, GitCheckoutTagInLocalRepo: [echo] [GIT] /shared/eclipse/e4/dwtest/build/e4/org.eclipse.releng.eclipsebuilder_R4_2_primary/../src/scmCache/git___git_eclipse_org_gitroot_platform_eclipse_platform_releng_git >> git checkout --force v20100222-1554 [exec] Note: checking out 'v20100222-1554'. [exec] [exec] You are in 'detached HEAD' state. You can look around, make experimental [exec] changes and commit them, and you can discard any commits you make in this [exec] state without impacting any branches by performing another checkout. [exec] [exec] If you want to create a new branch to retain commits you create, you may [exec] do so (now or later) by using -b with the checkout command again. Example: [exec] [exec] git checkout -b new_branch_name [exec] [exec] HEAD is now at 8323236... This commit was manufactured by cvs2svn to create branch 'R4_HEAD'. GitFetchFileFromLocalRepo: [copy] Warning: Could not find file /shared/eclipse/e4/dwtest/build/e4/src/scmCache/git___git_eclipse_org_gitroot_platform_eclipse_platform_releng_git/features/org.eclipse.sdk.examples-feature/feature.xml to copy. So, that's a step back. And now looking at detail at git web address from above, if I am reading it right, the tag actually went "way back" in time: -feature@org.eclipse.sdk.examples=GIT,tag=v20110912,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.git,path=features/org.eclipse.sdk.examples-feature -plugin@org.eclipse.sdk.examples=GIT,tag=v20110503,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.git,path=bundles/org.eclipse.sdk.examples +feature@org.eclipse.sdk.examples=GIT,tag=v20100222-1554,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.git,path=features/org.eclipse.sdk.examples-feature +plugin@org.eclipse.sdk.examples=GIT,tag=v20100222-1554,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.git,path=bundles/org.eclipse.sdk.examples To make matters worse, I was going to use the egit client to "inspect" current values in R4_HEAD (after saying "fetch from upstream") but when I look at history, Its is not obvious to me the version I'm seeing if from the e4Build-R42 (as the web page shows) and I see no obvious history that has the previous tag (v20110912 instead of v20100222-1554). So, that's another step back (probably in my lack of understanding?) And to top it all off, I accidentally blew away the "tweaked" get-release.sh Paul was so kind to help with ... pretty sure I can recontruct it. But ... something is obviously wrong ... with script? My understanding? both? My guess is there must me some git command line way to roll-back the whole commit, based on hash af46cf67ac971001b64e378a3f34949c635d702e ? Which, I'll have to read up on ... unless someone is up late and tells me how to do it :)
(In reply to comment #63) > But ... something is obviously wrong ... with script? My understanding? both? > > My guess is there must me some git command line way to roll-back the whole > commit, based on hash af46cf67ac971001b64e378a3f34949c635d702e ? Which, I'll > have to read up on ... unless someone is up late and tells me how to do it :) You can get rid of the e4Build commit one of 2 ways: Use git revert af46cf67ac971001b64e378a3f34949c635d702e to create a new commit which will undo the old one and push it. Use git reset --hard 97c08d63035b64d9097a4453efe791f5bbefbc91 and then use a force push ... which works unless we've disabled that :-) PW
Thanks Paul, I'll try the reset. I need another clarification I keep for getting to ask (or, I've asked, and forgotten the answer). Is the /shared/eclipse/e4 location "we" have been using to do the primary eclipse 4 build tests for exclusive use of "eclipse 4 primary builds"? Or is this really for eclipse incubator and we just sort of started testing there? I also see an "orion" directory under /shared/eclipse/e4 ... it that "the real" orion build location? Or something left over from testing? Seems current ... I'm just confused by naming/directory structure, so thought I'd ask. Seems it'd be better to have /shared/eclipse/orion /shared/eclipse/e4inc /shared/eclipse/eclipse4 /shared/eclipse/eclipse3 or something. Just asking for clarification. Not drastic change.
I meant to say, I'd try the softer "revert", first. But trying that on my local repo I get $ git revert af46cf67ac971001b64e378a3f34949c635d702e # On branch R4_HEAD nothing to commit (working directory clean) And, I do mean my local one, not the "working tree" created on build.eclipse.org during the build (which I have now deleted). I think did try $ git reset --hard 97c08d63035b64d9097a4453efe791f5bbefbc91 HEAD is now at 97c08d6 remove blank lines (or use comments) (which 'sounds right) But, you are fight, seems to we "deny fast forward" (which we fought so hard for :) $ git push --force Total 0 (delta 0), reused 0 (delta 0) remote: error: denying non-fast-forward refs/heads/R4_HEAD (you should pull first) To ssh://david_williams@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git ! [remote rejected] R4_HEAD -> R4_HEAD (non-fast-forward) error: failed to push some refs to 'ssh://david_williams@git.eclipse.org/gitroot/platform/eclipse.platform.releng.maps.git'
(In reply to comment #66) > I meant to say, I'd try the softer "revert", first. But trying that on my local > repo I get > > $ git revert af46cf67ac971001b64e378a3f34949c635d702e > # On branch R4_HEAD > nothing to commit (working directory clean) > > And, I do mean my local one, not the "working tree" created on > build.eclipse.org during the build (which I have now deleted). > Ok I got the revert and push to work (I think) by doing a pull first, on my local machine, from command line (I sure don't understand egit very well, obviously :) But, think I've restored the old but not so old maps, and will try another build.
Here's a theory as to what went wrong with the semi-autotagging script. When Paul and I were running it, part of the input was "the last I-build" and from memory we thought "the last I build" was I20120320-0800 but that was from 3.8 I build. (and then not quite the last one, which I doubt matters ... but ... from 3.8?) The last I build from 4.2 builds was I20120321-0610 Paul, would that have lead to the "weird" results?
(In reply to comment #68) > Paul, would that have lead to the "weird" results? It shouldn't. It's only used for the /bin/bash git-submission.sh line to generate a report (it's reading logs) and we commented that line out. PW
There's something else going on with the builder. It does things (destroying the repos) at .../build/e4/gitClones We should make sure there's nothing else writing there. Maybe it expects to write into the map location, and we need to copy the maps to the regular location (this doesn't sound correct, but something is writing there). PW
For what its worth, I have started to use /shared/eclipse/eclipse4 as the primary place to build eclipse 4, since seems to me the e4 directory is/was used for lots of things and I found it confusing. (and, getting close to wanting to "show" some builds, and didn't want it to be in "dwtest" :)
*** Bug 375920 has been marked as a duplicate of this bug. ***
Obviously this bug is now a blocker because we have no other builds.
For the hard core, bleeding edge Eclipse Project committers, I think the builds are working well enough that some "extra eyes" would help. So ... you can see the builds (sort of) by going to http://build.eclipse.org/eclipse/eclipse4/build/supportDir/ and picking the latest I<buildID> (or, if its blank, or you see just "buildlogs" then its probably one in progress or one that failed, so try the next to the latest). You should see a poorly formatted "index" page (poorly formatted, because, I'm assuming, merely because it is missing all the usual "theme" goodies it normally has on download.eclipse.org). For example, as of right now, there's a build index page at http://build.eclipse.org/eclipse/eclipse4/build/supportDir/I20120403-2044/ There's still a lot to do before putting these on "downloads", but if anyone has time to take a look and see if there are any gross errors (such as picking up complete wrong version/stream of things, let me know (i.e. open a releng bug, after checking the "depends on" list, to make sure not already open). Of interest are the compile errors in 2 of the "test bundles" where "split streams" are obviously needed, but there are bugs already open for those http://build.eclipse.org/eclipse/eclipse4/build/supportDir/I20120403-2044/testResults.php And, I have not even looked at trying to trying to run unit tests ... I think that's a week or so a way, at least. I tried one of the linux builds, and at least it came up! (which we only got to that point 4 or 5 hours ago ... so ... keep in mind ... even though "its building" ... it is is not yet "building correctly" (i.e. go in with low expectations and just look for really bad errors). There will still be alot of rapid change taking place (such as "location" of those builds might change). For now I'll just be kicking off builds manually 3 or 5 times per day. Perhaps early next week we'll be back to scheduled builds.
@David, Paul, John and others First let me say that you guys are doing a great job. This isn't an easy task and it seems like a pretty rough initiation into platform builds. But a working build is a good milestone. I checked out the build you pointed to and it launches on MacOS (64 bit). I checked the version numbers for the p2 bundles and they all match the last known good 4.2 build. The p2 UI comes up and I even managed to install something (which means that the build is producing a proper p2 profile). I noticed the splash screen has changed (it's not the same as the last known good 4.2 build). In particular, the logo has moved up (about 50px) which is the same as a previous splash. I realize the splash is the least of our concerns, but it may indicate that the wrong platform bundle is being picked up.
(In reply to comment #74) > There's still a lot to do before putting these on "downloads", but if anyone > has time to take a look and see if there are any gross errors (such as picking > up complete wrong version/stream of things, let me know (i.e. open a releng > bug, after checking the "depends on" list, to make sure not already open). I looked at I20120404-0051 and this sees to include old versions of the common bundles, e.g. JDT and PDE are neither from 'integration' nor 'master' branch. So, it looks like the 4.2 build is still taking a (old) 3.8 build and includes it, instead of building all together.
(In reply to comment #76) > > I looked at I20120404-0051 and this sees to include old versions of the common > bundles, e.g. JDT and PDE are neither from 'integration' nor 'master' branch. > So, it looks like the 4.2 build is still taking a (old) 3.8 build and includes > it, instead of building all together. We haven't run auto-tagging yet, so all of the map files point to the last 3.8 I build. We haven't picked up successful integration branch submissions. autotagging: bug 375807 PW
(In reply to comment #75) > I noticed the splash screen has changed (it's not the same as the last known > good 4.2 build). In particular, the logo has moved up (about 50px) which is > the same as a previous splash. I realize the splash is the least of our > concerns, but it may indicate that the wrong platform bundle is being picked > up. Thanks for taking a look Ian. It looks like our map entry is out of date for this one. We are picking up: plugin@org.eclipse.platform=GIT,tag=v20120203-1648 But the vertical alignment of the splash was changed in a commit on March 8. Once we get our tagging script going we should be getting the latest splash.
(In reply to comment #74) > For the hard core, bleeding edge Eclipse Project committers, I think the builds > are working well enough that some "extra eyes" would help. So ... you can see > the builds (sort of) by going to > > http://build.eclipse.org/eclipse/eclipse4/build/supportDir/ I just wanted to report that I picked up I20120404-0051 this morning and gave it a spin. Everything looks good to me. I verified all the split stream content had the right versions. Some of the tags are a bit stale as we know, but overall the build seemed perfectly usable. Great progress since last week, thanks for all your hard work David!!
I tried I20120403-2231 on Fedora 16 x86_64 and things look pretty good (excluding the other issues mentioned in previous comments)! Excellent work, David, Paul, and everyone else!
While this bug is technically about getting 4.2 I builds going on build.eclipse.org, I thought it worth documenting here (so I won't forget :) that at the arch/planning call today, the general consensus was to "get things working" in the following order or priorities: 4.2 I builds 4.2 N builds 3.8 I builds 4.2 Unit tests 3.8 Unit tests
(In reply to comment #81) > 4.2 I builds > 4.2 N builds > 3.8 I builds > 4.2 Unit tests > 3.8 Unit tests I assume 'produce p2 repositories' is also in there? Hopefully as an output of the different builds.
(In reply to comment #82) > I assume 'produce p2 repositories' is also in there? Hopefully as an output of > the different builds. Yes that's part of the builds - repos are a prerequisite to building the zip files. We do need to start promoting the builds to the usual 4.2-I-builds location though once we are happy with the build. Currently you can see them here: http://build.eclipse.org/eclipse/eclipse4/build/supportDir/updates/4.2-I-builds/
(In reply to comment #83) > Currently you can see them > here: > > http://build.eclipse.org/eclipse/eclipse4/build/supportDir/updates/4.2-I-builds/ More good news then. I was able to update (using p2) from a previous (Kim/IBM 4.2 build) to new (David/Foundation 4.2 build). To be precise, I updated from: Eclipse SDK 4.2.0.I20120314-1735 to Eclipse SDK 4.2.0.I20120404-0051 on MacOS. It look a little longer to get the artifacts that I would have thought (more downloads than I thought it should have been), but if our tagging / qualifiers isn't right then this is expected. Before we declare victory on this we should re-test this update. I'll use p2 to update my SDK so this will get some testing as builds get churned out.
While building sources today for the tag I20120405-0114, I have found following issues in map files: core.map fragment@org.eclipse.core.resources.win32=GIT,tag=v20100505-1235,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.resources.git,path=bundles/org.eclipse.core.resources.win32 there is no such bundle in the repo http://git.eclipse.org/c/platform/eclipse.platform.resources.git/tree/bundles feature.map all org.eclipse.pde.api.tools.ee* fragments have path ending with "-feature", while their actual location is different: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/tree/apitools feature org.eclipse.platform-feature has been moved from features to oldfeatures in repo, but the map file still uses the old location.
and testframework.map plugin@org.eclipse.test.dispatcher=GIT,tag=HEAD,repo=git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.git,path=bundles/org.eclipse.test.dispatcher lacks corresponding plugin.
(In reply to comment #83) > (In reply to comment #82) > > I assume 'produce p2 repositories' is also in there? Hopefully as an output of > > the different builds. > > Yes that's part of the builds - repos are a prerequisite to building the zip > files. We do need to start promoting the builds to the usual 4.2-I-builds > location though once we are happy with the build. Currently you can see them > here: > > http://build.eclipse.org/eclipse/eclipse4/build/supportDir/updates/4.2-I-builds/ In case any readers didn't notice from platform-releng-dev list, the "build machine location" for updates has been moved to http://build.eclipse.org/eclipse/eclipse4/siteDir/updates/4.2-I-builds/
One thing that's just occurred to me ... for Eclipse, SDK, etc., we've always had two download sites: for 3.8, the tranditional ...eclipse/downloads/drops and for 4.2 the newer ...eclipse/downloads/drops4 We'll continue that now what we will be building both "from scratch". But, what about for the Equinox downloads? We've have been using merely .../equinox/drops As far as I know, the bits should be the same for everything there when building 3.8 from scratch as well as 4.2 from scratch, right? So ... I suggest we just put the 4.2-from-scratch bits there on equinox/drops and "throw away" the pieces from 3.8-from-scratch bits. Let me know if anyone knows otherwise. Naturally, the ambitious may test on the build machine itself if the bits are actually the same, when there are two equinox builds that have exactly same input.
(In reply to comment #88) > So ... I suggest we just put the 4.2-from-scratch bits there on equinox/drops > and "throw away" the pieces from 3.8-from-scratch bits. +1.. they should be identical.
I'd like to count this bug as "fixed" for M7. We certainly do have 4.2 as primary build, so definitely the "blocking" aspect of it is fixed. There are still several issues with the build and tests, but think they are well tracked in other bugs 339430, 375654, 376460, 377365 and no need to keep this one open (as blocking).
