Bug 232641 - Invalid byte codes produced since M7
Summary: Invalid byte codes produced since M7
Status: VERIFIED INVALID
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 blocker (vote)
Target Milestone: 3.4 RC2   Edit
Assignee: Maxime Daniel CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 232588
  Show dependency tree
 
Reported: 2008-05-17 03:02 EDT by David Williams CLA
Modified: 2008-05-23 06:49 EDT (History)
3 users (show)

See Also:


Attachments
Exception logged indicating corrupt class file (9.64 KB, text/plain)
2008-05-17 03:04 EDT, David Williams CLA
no flags Details
current version of the jar file being produced (1.44 MB, application/octet-stream)
2008-05-17 03:06 EDT, David Williams CLA
no flags Details
M7 version of the jar file in question (where class file is "ok"). (1.44 MB, application/octet-stream)
2008-05-17 03:07 EDT, David Williams CLA
no flags Details
javap output for M7 class file file in question (28.29 KB, text/plain)
2008-05-17 03:09 EDT, David Williams CLA
no flags Details
javap output for current version of this class file in question (28.11 KB, text/plain)
2008-05-17 03:23 EDT, David Williams CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2008-05-17 03:02:39 EDT
We in WTP are at a loss. It appears we have some class files being produced that are invalid. This is originally reported in WTP in bug 232588. 

Funny thing is, we, in WTP, have not changed the JDT compiler since M7, though the platform itself may have. For M7 (and since) we have used the base builder tagged with M7_34, which I'd assume has the M7 version of the compiler. 

And, I'm assuming the class files are invalid since the Sun VM says they are. The IBM JRE does not say they are, but I don't think it has the same default 'verify' options, so it may not be checking the validity of the class file? 

The Exception says 
java.lang.ClassFormatError: Illegal field name "(Z)L;" in class org/eclipse/jst/pagedesigner/parts/DocumentEditPart

And, indeed there is a "(Z)L;" in the current class file (if I decompile using javap). Where as in the M7 version of the same class, there is no "(Z)L;". 

I'll attach the stack trace we get, and the jar that contains the DocumentEditPart, in case that helps analysis. 

I should note, I've tried many combinations of platform and wtp, and it does seem to be our class file that's different (bad) since M7, but our build infrastructure has not changed since then ... just the pre-reqs. 

We do compile against the classes in the PPC version of the platform, and are seeing these errors on Windows, so I'm also wondering if something  has gone wrong with the PPC version of the platform since M7?
Comment 1 David Williams CLA 2008-05-17 03:04:21 EDT
Created attachment 100757 [details]
Exception logged indicating corrupt class file
Comment 2 David Williams CLA 2008-05-17 03:06:00 EDT
Created attachment 100758 [details]
current version of the jar file being produced
Comment 3 David Williams CLA 2008-05-17 03:07:45 EDT
Created attachment 100759 [details]
M7 version of the jar file in question (where class file is "ok").
Comment 4 David Williams CLA 2008-05-17 03:09:03 EDT
Created attachment 100760 [details]
javap output for M7 class file file in question
Comment 5 David Williams CLA 2008-05-17 03:23:17 EDT
Created attachment 100761 [details]
javap output for current version of this class file in question

I should add, there's only a few cases where we've found "corrupt class files", if that is what's going on. It's not like a massive failure. But, enough to prevent some critical function from working, at least with Sun's VM. I'm not sure if IBM's VM would run completely correctly, or if the editor just opens because it doesn't verify the whole class? 

I'm opening this bug here hoping you can help us get to the bottom of it ... is there any debug techniques you'd recommend? 

Perhaps you could at least explain what 
Illegal field name "(Z)L;"
means and if indeed that sounds like a corrupt class file to you? 

Thanks.
Comment 6 David Williams CLA 2008-05-17 03:27:47 EDT
Kim, adding you to CC for a couple of questions: 

Did you (platform) switch compiler versions after you produced M7, or before? So you are now using the M7 version of the compiler? 

Also, can you think of anything that might be "different" about the PPC version of the platform since M7? 

thanks, 
Comment 7 David Williams CLA 2008-05-17 03:55:08 EDT
I'd be remiss not to include Thomas in on the fun as well ... for a while I was thinking of opening the bug in Equinox thinking there perhaps might be a recent change in the classloader that was the cause. 

I've since come to think that's not the case, but since the error does appear deep in a stack of class loading calls (and, shortly after a lock call, so bug 229621
had caught my eye for a while) ... so, figured I should at least point it out to that team. 
Comment 8 David Williams CLA 2008-05-17 04:05:13 EDT
I have just now confirmed from our log files, it is the exact same compiler being used in the good M7 build, and the current bad builds. 

Compiler Report

Compiler: Eclipse Java Compiler Version: 0.858, 3.4.0 milestone-7

Number of source files: 707 Number of classfiles: 954

Problems: 14 (Errors: 0 Warnings: 14 ) 



http://download.eclipse.org/webtools/downloads/drops/R3.0/S-3.0M7-20080506040630/compilelogs/plugins/org.eclipse.jst.pagedesigner_1.1.0.v200804232228/@dot.bin.html

Comment 9 Philipe Mulet CLA 2008-05-17 04:53:11 EDT
Would you have the source for the file: DocumentEditPart ?

Re: comment 6
I know Kim normally step to a new milestone compiler for building the next milestone. So ECJ M6 would be used to build M7, and ECJ M7 would be used to build RC1... 

Are there some tools massaging your classfiles after we produced them ?
Comment 11 Philipe Mulet CLA 2008-05-17 05:11:31 EDT
Comment 8 seems to rule out a regression in compiler though.
Comment 12 Philipe Mulet CLA 2008-05-17 05:15:11 EDT
Wondering if this couldn't be cause by pack200... is it in the picture somehow ?
Comment 13 Philipe Mulet CLA 2008-05-17 05:31:12 EDT
Re: comment 5
Indeed the disassembled output of the corrupted classfile shows it is very inconsistent. The constant pool seems completely ill-formed (the constant pool contains all the constant/signature etc... referenced from the bytecode).
Comment 14 Thomas Watson CLA 2008-05-17 07:44:59 EDT
(In reply to comment #7)
> I'd be remiss not to include Thomas in on the fun as well ... 

Thanks!!  :)

> for a while I was
> thinking of opening the bug in Equinox thinking there perhaps might be a recent
> change in the classloader that was the cause. 
> 
> I've since come to think that's not the case, but since the error does appear
> deep in a stack of class loading calls (and, shortly after a lock call, so bug
> 229621

This should not be the cause since the class loader locking strategy has remained unchanged by default.  We will not be able to move to class name locking by default until Sun fixes the VM class loader locking in JavaSE 7.

> had caught my eye for a while) ... so, figured I should at least point it out
> to that team. 
> 

Thanks, but in this case the class file does look bad according to comment 13.
Comment 15 David Williams CLA 2008-05-17 08:19:38 EDT
(In reply to comment #12)
> Wondering if this couldn't be cause by pack200... is it in the picture somehow
> ?
> 

It is in the picture, sort of. I can test/run some things to take it out of the picture completely to test that hypotheses (but, not till later this afternoon (eastern time). 

I say "sort of" since it would have to be during the conditioning stage (I've ruled out signing, and packing per se). 

Comment 16 Thomas Watson CLA 2008-05-17 08:56:19 EDT
Have you tried exporting the bundle from a workspace using M7 and running with that?  Wouldn't this also use JDT from M7 to build your bundle?
Comment 17 David Williams CLA 2008-05-17 15:09:57 EDT
Here's a report on the latest findings: 

Yes I can export the plugin (with M7 or I20080516081047) and glancing at the class file with javap, it seems ok (similar to the known "good" ones). 

Also, I can do PDE automated build on an x86 linux system that turns out ok (and that system has conditioning off and no signing). 

While looking to turn conditioning off on production machines, I did stumble upon a significant finding. 

The jars at our wst level have the expect eclipse.inf, where they say 
#Processed using Jarprocessor
pack200.args = -E4
pack200.conditioned = true

But the jars at our jst level do not say -E4, only 
#Processed using Jarprocessor
pack200.conditioned = true

The logic in our builds are exactly the same, so not sure why would level would be picking up the E4, and not the others, but that's what I'm investigating now. 
Comment 18 David Williams CLA 2008-05-17 19:56:28 EDT
All is well, thanks everyone for your comments and suggestions. 

As some of you suspected, this was a "pack200" problem, previously observed and discussed in bug 154069 (and bugs it points to). 

The work around for that infamous pack200 bug has always been down downgrade effort by specifying -E4 to pack200. 

And, I was trying to  :) 
but made a mistake (or, copied a mistake?) that somehow worked for some of our zip files (impossibly) but did not work for others. 

Once I corrected that bug in our build scripts, things seem whole again (the class file is not corrupt, and the jars have -E4 in the eclipse.inf files as expected). 

Thanks again. 
Comment 19 David Williams CLA 2008-05-17 20:38:31 EDT
FYI, I've opened bug 232679 as a "follow up" to this bug. (proposing that -E4 be the default for jarProcessor). 
Comment 20 Maxime Daniel CLA 2008-05-19 02:40:38 EDT
Moved to RESOLVED INVALID by the reporter. Reading the bug, I would agree (as far as JDT is concerned, and since a fup bug was opened for the remainder, we should be 100% safe).
Comment 21 Frederic Fusier CLA 2008-05-23 06:49:28 EDT
Verified for 3.4RC2