Bug 325311 - [comparator] ArrayIndexOutOfBoundsException in Comparator
Summary: [comparator] ArrayIndexOutOfBoundsException in Comparator
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.6   Edit
Hardware: PC Windows 7
: P3 major (vote)
Target Milestone: 3.7 M3   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-09-15 03:16 EDT by David Williams CLA
Modified: 2010-10-20 14:00 EDT (History)
4 users (show)

See Also:


Attachments
console log showing whole exception stack (6.52 KB, text/plain)
2010-09-15 03:16 EDT, David Williams CLA
no flags Details
problematic jar from WTP 3.2.2 release (808.15 KB, application/octet-stream)
2010-10-19 16:30 EDT, David Williams CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2010-09-15 03:16:26 EDT
Created attachment 178903 [details]
console log showing whole exception stack

I'll attach whole exception message, but in brief: 

java.lang.ArrayIndexOutOfBoundsException
at org.eclipse.equinox.p2.internal.repository.comparator.java.ClassFileStruct.u1At(ClassFileStruct.java:43)

This _appeared_ to be happening while "comparing" one of our test bundles: 
org.eclipse.jst.jsp.core.tests 

As in bug 325158 I will try excluding that specific bundle, to see if processing can continue. 

It is kind of especially bad, because even though I have "ignoreErrors=true", and indeed the comparing does continue, in the end the "content.jar/xml" file is empty of metadata. so, kind of deceptive. I'm assuming it is empty due to this exception occurring and not being handled at the right place?
Comment 1 David Williams CLA 2010-09-17 13:29:20 EDT
(In reply to comment #0)

> 
> It is kind of especially bad, because even though I have "ignoreErrors=true",
> and indeed the comparing does continue, in the end the "content.jar/xml" file
> is empty of metadata. so, kind of deceptive. I'm assuming it is empty due to
> this exception occurring and not being handled at the right place?

It appears the metadata not being created was due to bug 313615. That was fixed in RC2, and my "basebuilder" was still RC1. So I've moved to basebuilder 36_RC4 and hopefully that metadata error will not occur. 

Excluding the jar with the <exclude tag so if RC4 fixes the metadata part, I can probably use this comparator functionality but ... of course, I still think the underlying AIOOB problem should be fixed ... and I'd recommend to fix things so that _any_ such exception in comparator allows it to complete without adding it to "exclude" list (I've hit it now for 3 bundles, for two reasons ... so, seems common).
Comment 2 Andrew Niefer CLA 2010-10-14 18:14:10 EDT
It seems the trace you are seeing is actually the message on a ClassFormatException.  The ArrayIndexOutOfBounds apparently indicates a problem with the classfile.

We need to catch the ClassFormatException and return a human readable message with the class name.

I don't know if it is possible to get the specific thing that we think is the problem.

Also, this code was ported over from JDT, Olivier, do you know if there were any bugs fixed in the JDT that should be fixed here as well?
Comment 3 Olivier Thomann CLA 2010-10-15 08:48:17 EDT
I would look at it as soon as I get the corresponding .class files.
Comment 4 Olivier Thomann CLA 2010-10-19 15:40:59 EDT
David, could you please attach the .class files so that this issue can be investigated before the files are no longer available ?

Thanks.
Comment 5 David Williams CLA 2010-10-19 16:30:19 EDT
Created attachment 181222 [details]
problematic jar from WTP 3.2.2 release

Here is the org.eclipse.jst.jsp.core.tests jar from our WTP 3.2.2 release. 

I'm assuming the problem is in there. If not, I can try to reproduce on a recent build, to confirm the problem is still there, and if so, send more files?
Comment 6 Olivier Thomann CLA 2010-10-19 16:47:10 EDT
Andrew, please let me know where I can find the code of the latest comparator.
I'll check that as soon as I get it.
Comment 7 David Williams CLA 2010-10-19 16:57:18 EDT
FWIW, I'm using the version from the "basebuilder" that is tagged with R36_RC4. 
That is in the eclipse cvs repo, from project 
org.eclipse.releng.basebuilder
Comment 8 Andrew Niefer CLA 2010-10-19 16:59:17 EDT
I compared this jar against itself (using JarComparator#compare(File,File)) and got the ArrayIndexOutOfBoundsException on 
testfiles/bug183756/test-jar/bin/com/foo/TestTag.class

The latest version of the comparator code is in /cvsroot/rt, org.eclipse.equinox/p2/bundles/org.eclipse.equinox.p2.repository.tools
Comment 9 Olivier Thomann CLA 2010-10-19 18:17:30 EDT
I am investigating.
Comment 10 Olivier Thomann CLA 2010-10-20 09:20:50 EDT
This .class file seems to be corrupted for some reason. If I recompile the corresponding source, I can successfully disassemble it.
David, how is this class file built for the build ?
Is it prebuilt or is it compiled each time ?
Comment 11 David Williams CLA 2010-10-20 09:40:03 EDT
This jar is "prebuilt". It was obtained from a client/user to address some (other) specific bug ... so, not sure how it was created ... and probably not appropriate or worth "fixing" the class file. There is a comment in original bug, https://bugs.eclipse.org/bugs/show_bug.cgi?id=183756#c12 that indicates some issues with it were found before ... in the distant past! 

Perhaps this is a case where the comparator just needs to fail gracefully with some "invalid class file found" error message?

I'm adding Nick and Nitin, in case they have other insights into history or desired outcomes with their test jar.
Comment 12 Olivier Thomann CLA 2010-10-20 09:51:13 EDT
(In reply to comment #11)
> This jar is "prebuilt". It was obtained from a client/user to address some
> (other) specific bug ... so, not sure how it was created ... and probably not
> appropriate or worth "fixing" the class file. There is a comment in original
> bug, https://bugs.eclipse.org/bugs/show_bug.cgi?id=183756#c12 that indicates
> some issues with it were found before ... in the distant past! 
> 
> Perhaps this is a case where the comparator just needs to fail gracefully with
> some "invalid class file found" error message?
> 
> I'm adding Nick and Nitin, in case they have other insights into history or
> desired outcomes with their test jar.
I can easily provide a valid replacement for this .class file if needed. I wanted to be sure why this is an invalid .class file. If this is desired, then yes we need to add for more graceful failure in the comparator.
If not, then we should just fix it.
Comment 13 Olivier Thomann CLA 2010-10-20 12:24:01 EDT
Bug 325158 provides a better handling of boggus .class files.
The question remains to know if this .class file is corrupted on purpose. If not, it should be fixed. If yes, then the patch will take care of this and don't fail anymore.
Comment 14 David Williams CLA 2010-10-20 12:42:46 EDT
(In reply to comment #13)

> The question remains to know if this .class file is corrupted on purpose. 

Let's assume it is "on purpose", for now ... since that's the way we received it, and we should have the ability to accept "as is". (Its not something we deliver ... just use in a test case). It apparently sufficed to get to root of main bug, survive JUnit tests ... so I think to survive comparator is the only fix necessary at this time.
Comment 15 Andrew Niefer CLA 2010-10-20 14:00:06 EDT
Fixed with patch from bug 325158.