Bug 173651 - Add jar signing and packing
Summary: Add jar signing and packing
Status: VERIFIED FIXED
Alias: None
Product: Modeling
Classification: Modeling
Component: Releng (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Nick Boldt CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 217929
  Show dependency tree
 
Reported: 2007-02-09 09:56 EST by Nick Boldt CLA
Modified: 2008-05-06 16:27 EDT (History)
7 users (show)

See Also:


Attachments
console log showing failure to load gef source plugins (4.16 KB, text/plain)
2008-02-14 20:24 EST, Nick Boldt CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Nick Boldt CLA 2007-02-09 09:56:01 EST
Add support for pack200
Comment 1 Nick Boldt CLA 2008-02-07 02:25:23 EST
James, Christian:

Would either of you guys like to volunteer to try out the new signing & packing system? 

http://wiki.eclipse.org/Modeling_Project_Releng/Signing_And_Packing

Note that the packing step works but the update site is not yet taking advantage of it. Time to overhaul the update site generator.
Comment 2 Nick Boldt CLA 2008-02-08 01:43:20 EST
Bug renamed.

Wiki updated -- update site builder now works with Master zip and pack.gz files.

GEF is happy; JET packing (see bug 217929) is happy, signing TBD once I get committer status. 

I'm going to take on EMF next, after that it'll be UML2 or OQTV. Who wants some? ;-)
Comment 3 Christian Damus CLA 2008-02-08 07:38:33 EST
If it's working so well for everybody else, of course I want some of that!

Make it so!
Comment 4 James Bruck CLA 2008-02-08 09:24:09 EST
Me too please - for UML2.

Thanks,
- James.
Comment 5 Nick Boldt CLA 2008-02-14 17:17:16 EST
Max said:

  I’m having problems with some of the source plugins, org.eclipse.ocl.sources namely, not sure about others. When I install this plugin from the update site (M5 version), I have no zip files in it, and MANIFEST.MF is just a bunch of SHA1 digests. The plugin installs fine into eclipse, but seems to never be loaded. And this leads to unresolved dependencies in my build.
 Is that a known problem/with workaround or maybe I overlooked something?

I said:

  Crap. Sounds like something is screwy with the "master" zip I'm now using to generate the update site. I can revert to the old method that takes an SDK zip, but that might end up breaking the signing for features...

Can you try some other source feature/plugins? The signing/packing is now enabled for EMF, GEF, UML2, OCL, and JET.

I think I feel an M5huh coming on. :(
Comment 6 Nick Boldt CLA 2008-02-14 20:24:44 EST
Created attachment 89801 [details]
console log showing failure to load gef source plugins

Further testing shows:

* GEF: all features can be installed from update (including .pack.gz files) but the source features are invalid. Are the source plugins valid, I wonder? See attached console output.

* EMF: site OK for EMF 2.4.0M5, but not signed/packed (error in properties file); rerunning with property fixed, site.xml contains <category> but no <features>. This is bad.

* UML2: site.xml contains 2.2.0M5 <category> but no <features>. This, too, is bad.

* JET: site.xml contains 0.9.0M5 <category> but no <features>. Anyone seeing a pattern here?

* OCL: site.xml contains 1.2.0M5 <category> 4 times but only 3 blocks of <features>. Duplication is bad (but easily fixed). However, the 4th <category> w/ no matching <features> suggests a repeat of the above problem.

So, it appears we have two problems:

* source features should not be signed? or, were not jarred properly prior to signing

* buildUpdate.sh's new -siteXMLOnly option isn't working correctly with Master.zip contents to generate <features>, or is trying to put it in the wrong file (site-milestones.xml instead of site-interim.xml?)

The good news is that "unsigning" a feature should be easy, and shouldn't affect installation via zip as the zips don't include jarred features. In other words, I don't forsee the need for an set of M5huh builds, just a bunch of work to fix the update sites. Phew!
Comment 7 Anthony Hunter CLA 2008-02-14 21:13:39 EST
(In reply to comment #6)
> The good news is that "unsigning" a feature should be easy, and shouldn't
> affect installation via zip as the zips don't include jarred features. In other
> words, I don't forsee the need for an set of M5huh builds, just a bunch of work
> to fix the update sites. Phew!

Just to confirm, I did extract the GEF-ALL-3.4M5.zip and the source was fine.

Comment 8 Christian Damus CLA 2008-02-15 09:14:02 EST
(In reply to comment #7)
> 
> Just to confirm, I did extract the GEF-ALL-3.4M5.zip and the source was fine.

Similarly, source attachments in OCL M5 are alright, too, when installed from the SDK ZIP.

Note that the old style of source-attachment bundle still must be a directory bundle, in case the problem is an attempt to leave them JARred when installing from Update Manager.  The new system in PDE 3.4 uses individual JAR bundles.
Comment 9 Anthony Hunter CLA 2008-02-15 11:12:07 EST
(In reply to comment #7)
> 
> Just to confirm, I did extract the GEF-ALL-3.4M5.zip and the source was fine.
> 

Bad news, Christian checked and I have verified. The source is not working in GEF-SDK-3.4M5.zip.
Comment 10 Nick Boldt CLA 2008-02-15 17:05:23 EST
(In reply to comment #9)
> > Just to confirm, I did extract the GEF-ALL-3.4M5.zip and the source was fine.
> Bad news, Christian checked and I have verified. The source is not working in
> GEF-SDK-3.4M5.zip.

OK, the three SDK zips are fixed. Added the missing source plugins' MANIFEST.MFs and confirmed that the build process won't forget them next time.

Word to the wise, Ant's <zipfilesets> apparently need to be explicitly told to include third-level-nested files (eg., plugindir/META-INF/MANIFEST.MF) like this:
        <include name="**/org.eclipse.gef*" />
        <include name="**/org.eclipse.gef*/**" />
        <include name="**/org.eclipse.gef*/**/**" />

Zips and updated .md5 files are on emf.torolab and have been synched to download.eclipse.org. Give it 30 mins or so for mirror replication.

The GEF update site has been rejigged using the (older) non-signed/non-packed GEF-ALL-*.zip's contents instead of the new signed/packed Master zip... at least until I can figure out why the source features are being corrupted during signing (their manifests are being stripped of their Bundle-* directives and replaced with SHA1-Digest information; also, the src.zip contents are being purged). Spoke with Kim and the platform signing doesn't do this so maybe our source features are different... or maybe I shouldn't be manually jarring them before signing them. Unsure.

Now to roll this fix out to the other 4 projects...

Comment 11 Nick Boldt CLA 2008-02-15 20:36:54 EST
> Now to roll this fix out to the other 4 projects...

Other update sites have been rolled back to their non-signed/non-packed state. Actually, you might still get packed jars when installing, but NOT for source plugins.

Of course there's another problem I encountered, which is that UML2 won't install via update because it seems to think that it needs emf.converter 2.3.0 even though 2.4.0 is installed. See bug 219208. Not sure if this will block GMF builds, or is just an end-user issue.


Comment 12 Nick Boldt CLA 2008-02-23 01:07:44 EST
Signed source plugins are now fixed for EMF. Will run test build for GEF to verify, but it looks promising.
Comment 13 Nick Boldt CLA 2008-02-23 22:09:11 EST
(In reply to comment #12)
Signed GEF sources appear as expected in Help > Updates > Manage Config and Help > About > Plugins / Features.

If anyone wants to install and test using some deeper mechanism than just checking for breakages in in the above dialogs, you can get this latest test build here:

http://emf.torolab.ibm.com/gef/downloads/?project=gef&sortBy=date&hlbuild=0#latest 
http://emf.torolab.ibm.com/tools/gef/updates/site-interim.xml

Moving to verified. If you notice anything amiss, please reopen.
Comment 14 Christian Damus CLA 2008-05-06 16:18:16 EDT
Fix available in HEAD: 1.2.0M7 (S200805061053).
Comment 15 Christian Damus CLA 2008-05-06 16:19:41 EDT
Fix available in HEAD: 1.2.0M7 (S200805061125).
Comment 16 Christian Damus CLA 2008-05-06 16:25:18 EDT
Fix available in HEAD: 1.2.0M7 (S200805061205).
Comment 17 Christian Damus CLA 2008-05-06 16:27:57 EDT
Fix available in HEAD: 1.2.0M7 (S200805061125).