Bug 315919

Summary: finish P2izing the WTP build
Product: [WebTools] WTP Releng Reporter: David Williams <david_williams>
Component: relengAssignee: David Williams <david_williams>
Status: RESOLVED FIXED QA Contact: David Williams <david_williams>
Severity: normal    
Priority: P1 CC: kaloyan, neil.hauge
Version: 3.10Flags: david_williams: pmc_approved+
david_williams: pmc_approved? (raghunathan.srinivasan)
david_williams: pmc_approved? (naci.dai)
deboer: pmc_approved+
neil.hauge: pmc_approved+
kaloyan: pmc_approved+
Target Milestone: 3.10.0   
Hardware: PC   
OS: Windows 7   
Whiteboard: PMC_approved

Description David Williams CLA 2010-06-06 20:58:36 EDT
Our current build has kind of a mix of P2 oriented tasks and "old fashioned" deprecated methods of converting "update site" to p2 repo. 

I propose we "finish" this before the final build to make it a fully p2 oriented build. This does change the order of some steps so in theory could break something, but this is offset by a couple of factors ... 1) I've actually been using these p2 scripts in other builds (such as out patches build and wtp tools builds, and 2) there's a good test of the repo itself, by integrating with the Helios common site using the aggregator so if some checksum error or something is introduced, it will show up there. 

Not to mention, if there's unexpected issues, will be easy to revert to previous level of our builder.
Comment 1 David Williams CLA 2010-06-06 21:09:34 EDT
I think this deserves consideration as a last minute fix, since if we do it in maintenance stream, it will just introduce "changes" there, instead of making sure even the build scripts stay the same during maintenance. 

Concretely, I think the current repo we are creating contains a couple of little errors that this should fix. 1) many of the bundle names are not correctly externalized, in the content.jar/xml file since some of the deprecated P2 functions were not fixed in this respect. I think all the "OSGi default" externalization style are not correctly externalized in meta data. (They may be, after install, since the code that displays the name in installed version doesn't use the p2 metadata). 

2) some of the packed jars (the packed.gz artifacts) show the same size for "download" vs. "installed" sizes. I think the normal p2 publisher must do this correctly, but not the old deprecated ones. 

Neither of these problems are literally "stop ship" (since not much uses that data, currently) but since we plan to make this repo persistent "forever" it would be good not to have such obvious errors in it. Again, may make a funny difference between released repo and maintenance repo, that would be confusing.
Comment 2 David Williams CLA 2010-06-06 21:12:07 EDT
I've been running local tests today (Sunday) and if succeed could try a production build Monday. The only difference (only possibility of last minute surprise or error in production build) would be in the signing. That's something that's not done in my local builds. 

So far (doing last run now) the local builds look fine. Was a fairly easy transition, even though touches lots of lines in "customTargets.xml", they all follow a standard pattern that has been used before, in other builds.
Comment 3 David Williams CLA 2010-06-07 12:29:59 EDT
I'm going to give this a try on production machine. There is one complication though. Our "assembly features" are included in the buildRepo data. This is apparently desirable to some :) see bug 310674. This will likely take some extra steps near the end to "make out final repo" (which I haven't added yet). On my local builds, this shows up because many of the assembly features do not have correct copyrights or consistent licenses (they shouldn't have to have any, since not distributed). So those releng tests will fail, until I figure out easiest workarounds.
Comment 4 David Williams CLA 2010-06-08 01:24:51 EDT
Just cross reference, one side effect _might_ have been a jar that was not correctly packed so later shows up and invalidly signed. 

This _might_ have been some side effect of this change in build steps/procedure ... not sure. Just seems odd it'd show up now, after all this time, but after build steps changed. 

Documented it in bug 316088. Maybe we'll revisit someday and investigate causes, but for now just work around, by not packing.
Comment 5 David Williams CLA 2010-06-08 19:02:36 EDT
Seems all is well, building as before. I haven't checked/tested the actual categories at repo site yet (now created from category.xml files, instead of site.xml files) but should be similar to what it was (I noticed, as I was making this fix) there were a few missing in the site.xml file! 

Thanks for your support.