Community
Participate
Working Groups
Build ID: I20070608-1718 Steps To Reproduce: I am trying to install the JDT from a base platform runtime using the Europa discovery site and it is failing. I select the Eclipse Java Development Tools feature and go through the whole download procedure, then it tells me I am about to install a signed feature and when I click "Install" I get the following message (see attached screenshot): The content of this feature file has been corrupted This feature will not be installed. I have tried a few different mirrors and they all failed. However, when I click "Install All" instead of "Install" - the install goes without a hitch (keep in mind I am only installing one feature to begin with). I'm assuming there is something majorly wrong here? The behaviour should *definitely not* be different between buttons. I am using 3.3RC4. I'm assuming this is an Update thing, not a JDT thing - but I'll let you guys determine that. More information: I installed other features from the Europa site (Web Tools, etc.) but none of them were signed. Perhaps this is an Update + Signed Feature bug?
Created attachment 71508 [details] Screenshot showing error.
Moving to Platform/Update
Kim?
(In reply to comment #0) using the same Build id: I20070608-1718 go to Help --> Software Update --> Find and Install --> Search for new features Select the Europa Discovery Site --> Note: in the to_be_attached screen shot, you'll notice that there is not JDT feature to install RC4 comes with jdt.core v770, so I wonder why the updater would select v766 as an eligible update. Also sounds like a duplicate of bug 187396. As I noticed in this bug, the fact that JDT feature is proposed as an installable might be due to a problem with your Eclipse configuration. May you please try and reproduce this issue with a clean RC4 install?
Created attachment 71575 [details] [screen shot]: Europa Update Site
I think you are probably not seeing the JDT feature because you are starting with an "SDK" version of Eclipse - which already includes the JDT. My initial comment says to start with a "platform runtime".
Created attachment 71576 [details] JDT presented fine using OSU mirror; three screenshots + script used to run Eclipse Worked fine for me on Linux with IBM JDK 5.0 and clean workspace, cannot reproduce. See attached. I did notice however that the JDT core plugin is versioned/named org.eclipse.jdt.core_3.3.0.v_768.jar, but the other jdt plugins are more like org.eclipse.jdt_3.3.0.v20070601-1520.jar. See this highlighted in the third screenshot in the attached zip. Is there a timestamp missing? Shouldn't it be consistent will all the other ones naming scheme?
Note that org.eclipse.jdt.core_3.3.0.v_768.jar is one of the jars that I have found the org.eclipse.update.core.siteOptimizer to corrupt. I am working around this in the Europamatic per bug 187396 by copying the files-that-get-corrupted raw from their project-specific update sites. See the end of the log: http://dash.eclipse.org/~bfreeman/europa/createDigests-createDigests-log.html -- notice that org.eclipse.jdt.core_3.3.0.v_768.jar is copied from the platform's update site directly to the Europa update site because the siteOptimizer corrupts the jar when I pack200 everything.
(In reply to comment #8) Does this mean the Europa site is actually in better shape than the Eclipse Platform's Update site?
(In reply to comment #6) OK, missed that one
(In reply to comment #10) so, it sounds to me this one is a duplicate of bug 187396 (which has a normal priority). Also, may you please update the OS section of the bug to reflect the OS you're using.
(In reply to comment #7) see bug 193061 about incorrect version label. Please note that v768 is the kind of version label that is expected for jdt.core plug-in.
(In reply to comment #12) sorry, bug 193061 is INVALID, as I had a patched jdt.core JAR file
In response to comment #11: If you want to consider this is a dupe then I think a new bug should be spun off for the update mechanism that is failing. Sure - there may be a corrupt JAR here - but why does "Install All" simply seem to ignore it and work when "Install" does not? That seems pretty critical to me (the ability to install corrupt JARs) which is why I marked it as critical. And the OS field is up to date. It was up to date when I filed the bug. I'm running Windows XP. And shouldn't we be fixing corrupt JARs by fixing the siteOptimizer rather than trying to work around them? That seems pretty critical to me too.
I cannot reproduce this problem on my machine. I'm running the winxp drop on a with a 1.5 ibm vm. I used eclipse.org directory Mark, what vm are you using? Also, what mirror did you use? regarding comment #8 Different teams have different qualifier naming conventations. All that matters is that it's incrementing and following the versioning guidelines. regarding comment #9 The issue isn't that the platform milestone update site is wrong. The issue is that when the jar processor is run on the Europa site, zero size jars are created for jdt core jars. Bjorn has implemented a workaround that copies the correct jars from our update site to the Europa update site. note that bug 187396 is the issue with the standalone update. As an aside, I noticed that the /home/data/httpd/download.eclipse.org/releases/europa/plugins seems to contain all the plugins for all the milestones, not just the most recent milestone. So there are ~5500 plugins in the directory now from the last seven milestones/RCs. Shouldn't this be cleaned up after every milestone? I seem to remember David cleaning up the directory every time last year. Does the jar processor run over this directory every time or just in the staging area?
Hi Kim. I am running a Sun JRE - v1.5.0_06-b05. I had tried the following mirrors: [United States] Columbia University (http) [United States] crazeekennee.com (http) These two failed the same way for sure. I did try two others during my round of feature-fighting (I had other issues as well) but I won't include them as I am unsure if they were failing in this manner. I'll spend some time today trying to reproduce this for sure and perhaps even debug the update dialog a bit. We had a Europa-based release on Friday that I was trying to test so I was a bit brief in my bug report.
(In reply to comment #15) > Does the > jar processor run over this directory every time or just in the staging area? The jar processor runs only over the staging area and that is cleared before each Europa-matic build. See: http://dash.eclipse.org/~bfreeman/europa/prefix-prefix-log.html " Removing /home/data/users/bfreeman/europa/staging/features/, plugins/, and root files"
(In reply to comment #15) Kim, I reproduced the issue: - downloaded and installed the base platform runtime eclipse-platform-3.3RC4-win32 - went to Help --> Software Update --> Find and Install --> Search for new features and selected the Europa Discovery Site - selected the Eclipse JDT feature - downloaded all JAR's OK - once the signed jar install page is passed, install will either: -> complete ok if install all selected -> fail if install is selected: failure about a corrupted jdt.core v768 JAR file nothing in the .log
(In reply to comment #15) I can reproduce this problem on my machine and, better yet, from the Eclipse Project update site (not the Europa update site). I'm running WinXP with Sun JDK 1.6.0_01-b06. I installed the eclipse-platform-SDK-3.3RC4-win32.zip. I used the Platform's update site (http://download.eclipse.org/eclipse/updates/3.3milestones/site.xml) on the eclipse.org servers. I installed "Eclipse Java Development Tools 3.3.0.v20070531-1300-7o7jCHEFpPooeYp2xpl1ZR". When I choose "Install", I get "The contents of this feature file has been corrupted". If I start again (fresh Eclipse install) and choose "Install All", it works without any errors.
Dejan, I have replicated this issue. Why would "install all" work but not "install"? Are the the jars verified differently using each option?
Kim, I don't know the code in question well, but I will speculate that 'Install All' skips the digital signature checks while 'Install' will do these checks for each feature in question, thus revealing the corrupted signed archive the moment it tries to read it.
To replicate this issue, I unpacked the packed jar from the update site /builds/I200706151200/jdk/linuxppc/ibm-java2-ppc-50/jre/bin/unpack200 org.eclipse.jdt.core_3.3.0.v_768.jar.pack.gz org.eclipse.jdt.core_3.3.0.v_768.jar then verified it /builds/I200706151200/jdk/linuxppc/ibm-java2-ppc-50/bin/jarsigner -verify org.eclipse.jdt.core_3.3.0.v_768.jar jarsigner: java.lang.SecurityException: SHA1 digest error for org/eclipse/jdt/core/dom/InternalASTRewrite.class Andrew, any suggestions on why this may be happening. I replicated the same issue with the latest rebuild with doc fixes - I20070415-1200. An example jar is http://download.eclipse.org/eclipse/updates/3.3milestones/plugins/ org.eclipse.jdt.core_3.3.0.v_768.jar.pack.gz
So I have done some debugging here and found the source of the problem. The bug is on line #103 of org.eclipse.update.internal.ui.security.JarVerificationService. The code is a little convoluted because the JarVerificationPage is used as both a general information page as well as an error dialog. It is shown for each feature when the user clicks "Install", but is only shown once (for the first feature) when the user clicks "Install All", thus the user is never "prompted" with the corruption error. I would submit a patch (and I think in general this is bad code, but it is intertwined and I am leery to start ripping stuff out and inserting exceptions), but I am a little unsure as to how the update people think this should be handled. Should the user still be allowed to install a seemingly-corrupt feature, or should it bail all the time whenever a verification error occurs? Since this code is used in such a non-ideal (I would argue bad) way - in particular, both prompting the user with information and "prompting" the user with an error forcing them to cancel out of the install) - I can't create a patch without answering this question. If the code should break out of auto-pilot mode on a corruption error, then it probably should do the same when an "unknown" error occurs (or any other types of verification errors that may occur, for that matter). This could either be fixed in JarVerificationService#prompt(), or it could be looked for and addressed org.eclipse.update.core.Feature#verifyReferences() (around line #972), depending on how you want to handle this.
Mark, thanks for the debugging and we can try to fix this for 3.3.1, but for 3.3 we need to fix the initial problem (corrupted archive).
That sounds like the right thing to do for now. I'll take a shot at a patch if you have recommendations on what to do on verification errors. It would make sense to me to have it always fail in the event of any verification error but I don't know much about JAR signing and verification. There may be cases where you may want to be able to proceed anyway and force an unverified peg into a round hole. ;)
I do not yet understand what is going wrong with this jar. I took org.eclipse.jdt.core_3.3.0.v_770.jar from the latest build. Running pack200/unpack200 on it verifies fine. Running the siteoptimizer to pack/unpack results in the exception for InternalASTRewrite.class. This is the only class that is bad in the resulting jar, if I delete it, the jar verifies fine. The resulting InternalASTRewrite.class is completely different from the original (the kind of differences I might expect if it hadn't been conditioned in the first place).
This is the log from the jar processor from the last build we ran toward 3.3 Processing /builds/I200706151200/org.eclipse.releng.eclipsebuilder/../src/packtmp/eclipse-master-I20070615-1200.zip Extracting eclipse/plugins/org.eclipse.jdt.core_3.3.0.v_770.jar Running Pack on I20070615-1200/temp_eclipse-master-I20070615-1200.zip/eclipse/plugins/org.eclipse.jdt.core_3.3.0.v_770.jar Processing nested file: jdtCompilerAdapter.jar Children of /builds/I200706151200/src/I20070615-1200/temp_eclipse-master-I20070615-1200.zip/temp.org.eclipse.jdt.core_3.3.0.v_770.jar/jdtCompilerAdapter.jarare excluded from processing. Adding eclipse/plugins/org.eclipse.jdt.core_3.3.0.v_770.jar.pack.gz to I20070615-1200/eclipse-master-I20070615-1200.zip.temp There is an eclipse.inf for the antadapter portion of this jar, but that's not where the problem class resides jarprocessor.exclude.sign=true jarprocessor.exclude.children=true
We still don't know what is going wrong with this class file. Two workarounds exist, both involve adding something to org.eclipse.jdt.core/META-INF/eclipse.inf 1) If there is something special about this class, adding pack200.args = -E4,-Porg/eclipse/jdt/core/dom/InternalASTRewrite.class fixes the problem 2) adding "jarprocessor.exclude.children=true" also fixes the problem. We need to determine if there are any other jars where this is a problem. This seems to be a problem for org.eclipse.birt.report.engine_2.2.0.v20070614.jar, which suggests this may be related to the 0 sized pack.gz files from bug 187396.
I could update my platform install to all the other platform/pde/equinox project features without this error occuring.
Please see bug 193568 for a tool to use to check your update sites for pack.gz files that don't verify after unpacking.
Andrew discovered that there are problems with the packed jars where there are class files at the root of the plugin and and nested jars. The signature on a single class was corrupt within this packed jar was corrupt. When "Install" was selected in update manager, the signature didn't verify and the update of the feature failed. To avoid this problem, I have opened bug 193570 against jdt core to excluded child jars from being signed because this plugins matches the shape described above. Yesterday, Andrew wrote a tool to verify the signatures of packed jars. I have run this tool against the entire Europa staging site and only one other plugin exhibited this behaviour, org.eclipse.birt.report.engine I'll be opening a bug against BIRT and sending a note to the cross project list with an explanation of the problem. Also, Andrew has posted his tool to a bug 193568. Most plugins do not have this shape. This issue only impacted two of the eclipse plugins in the entire Europa update site.
As a side note, Olivier looked at the InternalASTRewrite class file. It is a perfectly valid java class file, the difference is the constant pool has been reorganized so the signing is broken. It is as if that one class file was never conditioned .
BIRT bug is 193587
Verified updating from platform with 1.5 vm to the jdt-feature from build I20070621-1340 update site. I'll submit this build to the Europa build.
