Bug 237166 - [compiler] ArrayIndexOutOfBoundsException thown rebuilding workspace after switching targets
Summary: [compiler] ArrayIndexOutOfBoundsException thown rebuilding workspace after sw...
Status: VERIFIED NOT_ECLIPSE
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.5 M1   Edit
Assignee: Kent Johnson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 237835 (view as bug list)
Depends on: 237314
Blocks:
  Show dependency tree
 
Reported: 2008-06-13 17:30 EDT by Alex Fitzpatrick CLA
Modified: 2008-08-06 15:15 EDT (History)
4 users (show)

See Also:


Attachments
Screencap of June 3 change (269.42 KB, image/jpeg)
2008-06-24 11:23 EDT, Alex Fitzpatrick CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Fitzpatrick CLA 2008-06-13 17:30:35 EDT
Build ID: 34RC4

Steps To Reproduce:
With a large number of projects in the workspace, using the workspace preferences to switch targets.

This leaves many, if not all, of the projects in the workspace with build errors and is difficult to recover from.

Wouldn't this kind of error on an array during a sort indicate multi-threading issues?


More information:
java.lang.ArrayIndexOutOfBoundsException
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.sortFields(ReferenceBinding.java:153)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.getField(SourceTypeBinding.java:858)
at org.eclipse.jdt.internal.compiler.lookup.Scope.findField(Scope.java:850)
at org.eclipse.jdt.internal.compiler.lookup.MethodScope.findField(MethodScope.java:354)
at org.eclipse.jdt.internal.compiler.lookup.BlockScope.getBinding(BlockScope.java:472)
at org.eclipse.jdt.internal.compiler.ast.QualifiedNameReference.resolveType(QualifiedNameReference.java:979)
at org.eclipse.jdt.internal.compiler.ast.Assignment.resolveType(Assignment.java:192)
at org.eclipse.jdt.internal.compiler.ast.Expression.resolve(Expression.java:882)
at org.eclipse.jdt.internal.compiler.ast.Block.resolve(Block.java:101)
at org.eclipse.jdt.internal.compiler.ast.IfStatement.resolve(IfStatement.java:233)
at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolveStatements(AbstractMethodDeclaration.java:444)
at org.eclipse.jdt.internal.compiler.ast.ConstructorDeclaration.resolveStatements(ConstructorDeclaration.java:466)
at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolve(AbstractMethodDeclaration.java:403)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1096)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1184)
at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.resolve(CompilationUnitDeclaration.java:535)
at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:743)
at org.eclipse.jdt.internal.compiler.ProcessTaskManager.run(ProcessTaskManager.java:137)
at java.lang.Thread.run(Thread.java:735)
Comment 1 Kent Johnson CLA 2008-06-16 10:31:55 EDT
Are you running on a dual core machine ?

Did the next full build finish successfully ?


We do not actually have multiple threads updating the fields of a source type, so this walkback is very unexpected.
Comment 2 Alex Fitzpatrick CLA 2008-06-16 10:41:12 EDT
Yes, running on a dual core.

No the next build fails. Once in this state it takes a lot of effort to get out of it.

However, the only time I see it consistently is when I switch targets.

When I say large workspace, I have 172 projects, most of them plugins and we have dependencies on JDT, PDE, EMF, GMF and even a few on WTP
Comment 3 Alex Fitzpatrick CLA 2008-06-16 10:41:57 EDT
And I'm not the only person seeing it, but I don't have statistics for you.
Comment 4 Kent Johnson CLA 2008-06-16 12:40:59 EDT
So which target are you switching from & to ?

Which VM are you running on ? Does a different VM make a difference ?
Comment 5 Alex Fitzpatrick CLA 2008-06-16 13:35:35 EDT
Probably caused by 237314
Comment 6 Alex Fitzpatrick CLA 2008-06-16 18:06:23 EDT
After the race condition PDE is adressed in 237314 the bounds exception no longer occurs.

However this does imply that there must be a race condition in ReferenceBinding, some sort of synchronization would not go amiss here.
Comment 7 Kent Johnson CLA 2008-06-17 10:08:59 EDT
But we do not have multiple threads fighting over the fields slot.

Or at least we don't believe we do.
Comment 8 Christian Vogt CLA 2008-06-19 14:11:56 EDT
Encountering the same issue. Here's a new stack trace:

java.lang.ArrayIndexOutOfBoundsException
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.sortFields(ReferenceBinding.java:153)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.fields(SourceTypeBinding.java:612)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.faultInTypesForFieldsAndMethods(SourceTypeBinding.java:594)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInTypes(CompilationUnitScope.java:438)
at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:736)
at org.eclipse.jdt.internal.compiler.ProcessTaskManager.run(ProcessTaskManager.java:137)
at java.lang.Thread.run(Thread.java:735)
Comment 9 Kent Johnson CLA 2008-06-24 10:28:32 EDT
I have been unable to duplicate any of the 4 stack traces we've seen since last week with a workspace consisting of RMP, RUML & RUMV.

I'll try again today with the VM from https://bugs.eclipse.org/bugs/show_bug.cgi?id=237835#c5
Comment 10 Alex Fitzpatrick CLA 2008-06-24 10:41:42 EDT
What kind of hardware are you testing with? I've seen this on a T60p (Centrino duo processor) with both J9 1.6 and sun 1.6
Comment 11 Alex Fitzpatrick CLA 2008-06-24 10:43:36 EDT
And I got another merge sort issue yesterday... this time from platform:

java.lang.ArrayIndexOutOfBoundsException
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at org.eclipse.ui.internal.views.markers.CachedMarkerBuilder.sortAndMakeCategories(CachedMarkerBuilder.java:1166)
at org.eclipse.ui.internal.views.markers.CachedMarkerBuilder.buildAllMarkers(CachedMarkerBuilder.java:236)
at org.eclipse.ui.internal.views.markers.CachedMarkerBuilder$1.run(CachedMarkerBuilder.java:353)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Comment 12 Alex Fitzpatrick CLA 2008-06-24 11:06:34 EDT
Is this the rogue thread?

java.lang.ArrayIndexOutOfBoundsException
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.sortMethods(ReferenceBinding.java:160)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.methods(SourceTypeBinding.java:1115)
at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.unResolvedMethods(ReferenceBinding.java:1223)
at org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.computeInheritedMethods(MethodVerifier.java:551)
at org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.computeInheritedMethods(MethodVerifier.java:493)
at org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:858)
at org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.verify(MethodVerifier15.java:781)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.verifyMethods(SourceTypeBinding.java:1662)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.verifyMethods(CompilationUnitScope.java:794)
at org.eclipse.jdt.internal.compiler.Compiler.resolve(Compiler.java:856)
at org.eclipse.jdt.internal.compiler.Compiler.resolve(Compiler.java:904)
at org.eclipse.jdt.internal.core.CompilationUnitProblemFinder.process(CompilationUnitProblemFinder.java:182)
at org.eclipse.jdt.internal.core.CompilationUnitProblemFinder.process(CompilationUnitProblemFinder.java:243)
at org.eclipse.jdt.internal.core.ReconcileWorkingCopyOperation.makeConsistent(ReconcileWorkingCopyOperation.java:190)
at org.eclipse.jdt.internal.core.ReconcileWorkingCopyOperation.executeOperation(ReconcileWorkingCopyOperation.java:89)
at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:709)
at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:770)
at org.eclipse.jdt.internal.core.CompilationUnit.reconcile(CompilationUnit.java:1224)
at org.eclipse.jdt.internal.core.CompilationUnit.reconcile(CompilationUnit.java:1185)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.reconcile(JavaReconcilingStrategy.java:131)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.access$0(JavaReconcilingStrategy.java:108)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy$1.run(JavaReconcilingStrategy.java:89)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.reconcile(JavaReconcilingStrategy.java:87)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.reconcile(JavaReconcilingStrategy.java:149)
at org.eclipse.jdt.internal.ui.text.CompositeReconcilingStrategy.reconcile(CompositeReconcilingStrategy.java:86)
at org.eclipse.jdt.internal.ui.text.JavaCompositeReconcilingStrategy.reconcile(JavaCompositeReconcilingStrategy.java:102)
at org.eclipse.jface.text.reconciler.MonoReconciler.process(MonoReconciler.java:77)
at org.eclipse.jface.text.reconciler.AbstractReconciler$BackgroundThread.run(AbstractReconciler.java:206)
Comment 13 Alex Fitzpatrick CLA 2008-06-24 11:23:01 EDT
Created attachment 105718 [details]
Screencap of June 3 change

Is this change possibly related?
Comment 14 Kent Johnson CLA 2008-06-24 11:30:44 EDT
I have a multi(2)-processor machine, not a duo-core laptop.

Alex, these are the 5th & 6th different walkbacks that all cause AIOOBE in
Arrays.mergeSort().

In most of the 6 cases, there is only 1 thread involved in computing the
fields/methods of the type.

We haven't heard about this from anyone else but you guys can reproduce these
fairly consistently.

6 different stack traces that all fail in the same place in the base class
libraries and in most cases the calling code hasn't changed in over a year.

The probability that this isn't a VM issue is 0.
Comment 15 Alex Fitzpatrick CLA 2008-06-24 11:51:14 EDT
Point taken, but we've reproduced this on three different VMs, J9 1.5, J9 1.6 and sun 1.6.

Higher incidence of cosmic rays at Palladium perhaps?

I'm still betting on a race condition in JDT and elsewhere. 

Comment 16 Kent Johnson CLA 2008-06-24 12:02:19 EDT
I hear you, but everyone else on JDT (> 8 developers) are all using duo-core laptops that are the same or similar to yours.

And not 1 of them has seen any race conditions that cause these AIOOBEs.
Comment 17 Alex Fitzpatrick CLA 2008-06-24 12:28:09 EDT
Nor do JDT developers have 1400 plugins in their target and 170 plugin projects from CVS in their workspace.

We clearly have an issue, none of us know the cause. There may be issues here that will impact our enterprise customers. The sooner we can identify the cause and the remedy the better.

So what do we try next?
Comment 18 Gary Johnston CLA 2008-07-08 21:04:15 EDT
I'm getting what looks like the same thing.  Happens intermittently (about 10-15 times over the course of a work day) while I'm doing "normal" Java development (i.e., no switching workspaces or target platform).  Have to turn off auto-build, clean all, restart workbench, rebuild all to recover.  I'm running 3.4 on a ThinkPad T42p (so only one processor) on Win XP Pro SP2.  I have 47 plug-in projects in my workspace.

Java version info:
  java version "1.6.0"
  Java(TM) SE Runtime Environment (build pwi3260sr1-20080416_01(SR1))
  IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 Windows XP x86-32 jvmwi3260-200804
15_18762 (JIT enabled, AOT enabled)
  J9VM - 20080415_018762_lHdSMr
  JIT  - r9_20080415_1520
  GC   - 20080415_AA)
  JCL  - 20080412_01

Typical stack trace:
java.lang.ArrayIndexOutOfBoundsException
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.mergeSort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at java.util.Arrays.sort(Unknown Source)
at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.sortMethods(ReferenceBinding.java:160)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.getExactMethod(SourceTypeBinding.java:789)
at org.eclipse.jdt.internal.compiler.lookup.Scope.findExactMethod(Scope.java:771)
at org.eclipse.jdt.internal.compiler.lookup.Scope.getMethod(Scope.java:2108)
at org.eclipse.jdt.internal.compiler.ast.MessageSend.resolveType(MessageSend.java:432)
at org.eclipse.jdt.internal.compiler.ast.MessageSend.resolveType(MessageSend.java:344)
at org.eclipse.jdt.internal.compiler.ast.Expression.resolve(Expression.java:882)
at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolveStatements(AbstractMethodDeclaration.java:444)
at org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.resolveStatements(MethodDeclaration.java:191)
at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.resolve(AbstractMethodDeclaration.java:403)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1096)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.resolve(TypeDeclaration.java:1184)
at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.resolve(CompilationUnitDeclaration.java:535)
at org.eclipse.jdt.internal.compiler.Compiler.resolve(Compiler.java:859)
at org.eclipse.jdt.internal.compiler.Compiler.resolve(Compiler.java:904)
at org.eclipse.jdt.internal.core.CompilationUnitProblemFinder.process(CompilationUnitProblemFinder.java:182)
at org.eclipse.jdt.internal.core.CompilationUnitProblemFinder.process(CompilationUnitProblemFinder.java:243)
at org.eclipse.jdt.internal.core.ReconcileWorkingCopyOperation.makeConsistent(ReconcileWorkingCopyOperation.java:190)
at org.eclipse.jdt.internal.core.ReconcileWorkingCopyOperation.executeOperation(ReconcileWorkingCopyOperation.java:89)
at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:709)
at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:770)
at org.eclipse.jdt.internal.core.CompilationUnit.reconcile(CompilationUnit.java:1224)
at org.eclipse.jdt.internal.core.CompilationUnit.reconcile(CompilationUnit.java:1185)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.reconcile(JavaReconcilingStrategy.java:131)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.access$0(JavaReconcilingStrategy.java:108)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy$1.run(JavaReconcilingStrategy.java:89)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.reconcile(JavaReconcilingStrategy.java:87)
at org.eclipse.jdt.internal.ui.text.java.JavaReconcilingStrategy.reconcile(JavaReconcilingStrategy.java:149)
at org.eclipse.jdt.internal.ui.text.CompositeReconcilingStrategy.reconcile(CompositeReconcilingStrategy.java:86)
at org.eclipse.jdt.internal.ui.text.JavaCompositeReconcilingStrategy.reconcile(JavaCompositeReconcilingStrategy.java:102)
at org.eclipse.jface.text.reconciler.MonoReconciler.process(MonoReconciler.java:77)
at org.eclipse.jface.text.reconciler.AbstractReconciler$BackgroundThread.run(AbstractReconciler.java:206)

Comment 19 Kent Johnson CLA 2008-07-09 09:52:17 EDT
Gary - please run with a Sun VM for a couple days to see if you reproduce the same problem.

thx
Comment 20 Gary Johnston CLA 2008-07-09 10:38:38 EDT
(In reply to comment #19)
> Gary - please run with a Sun VM for a couple days to see if you reproduce the
> same problem.
> 
> thx
> 

Will do.
Comment 21 Alex Fitzpatrick CLA 2008-07-09 11:22:36 EDT
After losing three hours to this one day last week I switched FROM Sun 1.6 to IBM J9 1.5 and I have not seen it since.

If I ever reproduce this with 1.5 I will let you know.
Comment 22 Kent Johnson CLA 2008-07-15 15:50:29 EDT
Gary, Alex & anyone else still listening - are you still seeing the same problems with other VMs ?
Comment 23 Alex Fitzpatrick CLA 2008-07-15 15:58:58 EDT
I have not seen this since I switched back to J9 1.5
Comment 24 Gary Johnston CLA 2008-07-15 16:04:11 EDT
(In reply to comment #22)
Thanks for the reminder.
I have *not* seen the problem since switching to a Sun 1.6 VM:
  java version "1.6.0_06"
  Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
  Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode, sharing)
Comment 25 Kent Johnson CLA 2008-07-16 10:09:21 EDT
Ok - so for anyone else that encounters this problem & can reproduce it 'regularly' - please attach your VM details to this bug.


Complaints have come in against this VM :

java version "1.6.0"
  Java(TM) SE Runtime Environment (build pwi3260sr1-20080416_01(SR1))
  IBM J9 VM (build 2.4, J2RE 1.6.0 IBM J9 2.4 Windows XP x86-32

and also this one :

java version "1.5.0"
  Java(TM) 2 Runtime Environment (build pwi32dev-20070511(SR5))
  IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32
Comment 26 John Arthorne CLA 2008-07-16 14:41:04 EDT
*** Bug 237835 has been marked as a duplicate of this bug. ***
Comment 27 Olivier Thomann CLA 2008-08-06 15:15:29 EDT
Verified for 3.5M1 using I20080805-1307