Bug 73226 - [VM] IllegalMonitorStateException in ReadWriteMonitor.exitRead
Summary: [VM] IllegalMonitorStateException in ReadWriteMonitor.exitRead
Status: RESOLVED INVALID
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.0   Edit
Hardware: PC All
: P3 major (vote)
Target Milestone: 3.1 M3   Edit
Assignee: Michael T. Brown CLA
QA Contact:
URL:
Whiteboard:
Keywords: vm
: 73375 73982 74794 76137 82283 (view as bug list)
Depends on:
Blocks: 73603
  Show dependency tree
 
Reported: 2004-09-03 05:33 EDT by Dani Megert CLA
Modified: 2005-03-06 22:11 EST (History)
13 users (show)

See Also:


Attachments
J9 VM dump (831.65 KB, text/plain)
2004-09-03 05:34 EDT, Dani Megert CLA
no flags Details
console output (130.66 KB, text/plain)
2004-09-20 11:54 EDT, Markus Keller CLA
no flags Details
stackdump (771.56 KB, text/plain)
2004-09-20 11:56 EDT, Markus Keller CLA
no flags Details
System events captured by DebugView (27.82 KB, text/plain)
2004-09-20 11:56 EDT, Markus Keller CLA
no flags Details
stackdump just before exception is thrown (815.47 KB, text/plain)
2004-09-20 13:43 EDT, Markus Keller CLA
no flags Details
dumps20040923-1011 (152.29 KB, application/zip)
2004-09-23 04:27 EDT, Markus Keller CLA
no flags Details
dumps20040924-0920 (142.04 KB, application/zip)
2004-09-24 03:42 EDT, Markus Keller CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dani Megert CLA 2004-09-03 05:33:13 EDT
I20040901 J9 JXE enabled

Happened after clicking Open Type. The UI is deadlocked (see attached dump).

java.lang.IllegalMonitorStateException
        at java.lang.Object.notifyAll(Native Method)
        at
org.eclipse.jdt.internal.core.search.indexing.ReadWriteMonitor.exitRead(Unknown
Source)
        at org.eclipse.jdt.internal.core.search.PatternSearchJob.search(Unknown
Source)
        at org.eclipse.jdt.internal.core.search.PatternSearchJob.execute(Unknown
Source)
        at
org.eclipse.jdt.internal.core.search.processing.JobManager.performConcurrentJob(Unknown
Source)
        at org.eclipse.jdt.core.search.SearchEngine.searchAllTypeNames(Unknown
Source)
        at org.eclipse.jdt.internal.corext.util.AllTypesCache.search(Unknown Source)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache$TypeCacher.doSearchTypes(Unknown
Source)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache$TypeCacher.run(Unknown Source)
Comment 1 Dani Megert CLA 2004-09-03 05:34:23 EDT
Created attachment 14391 [details]
J9 VM dump
Comment 2 Dani Megert CLA 2004-09-03 05:36:10 EDT
Not deadlocked: I can click "Cancel". However, when I click Open Type again I
get the progress dialog (Searching...) and it blocks again.

Nothing in .log (the IllegalMonitorStateException was in the console)
Comment 3 Kent Johnson CLA 2004-09-07 14:44:39 EDT
*** Bug 73375 has been marked as a duplicate of this bug. ***
Comment 4 Kent Johnson CLA 2004-09-08 04:41:40 EDT
Olivier, which VM are you using?
Comment 5 Kent Johnson CLA 2004-09-08 10:51:26 EDT
The current suspicion is a J9 issue.

Please check your log frequently to see how often you encounter this problem & 
update your VM daily.
Comment 6 Markus Keller CLA 2004-09-15 06:34:21 EDT
I just run into this bug with
"IBM J9SE VM (build 2.2, J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 j9n142-20040914",
(without JXE). I experience this about every second day.

Bug 73254 is a dup of this one. Bug 73603 looks also similar (->J9 bug?).
Comment 7 Jerome Lanneluc CLA 2004-09-15 10:08:12 EDT
*** Bug 73982 has been marked as a duplicate of this bug. ***
Comment 8 Michael T. Brown CLA 2004-09-16 13:55:22 EDT
By adding this:

-Xdump:heap+java:events=throw,filter=java/lang/IllegalMonitorStateException

to the command line will have J9 do a complete dump on an 
IllegalMonitorStateException. 

BTW, can someone provide a bit more info about this bug? When did it first 
start showing up? The first seems to be Sep 1. Is that really the earliest VM? 
Do you guys grab a new 2.2 VM daily? Do you grab a new Eclipse daily? i.e. what 
changed between everything's good, to now we see it?

Does it happen 2 mins into a run.. or if you have Eclipse up for 5 days then it 
happens.. I'm assuming it's seen on a uniprocessor box?

Stuff like that.
 
Comment 9 Olivier Thomann CLA 2004-09-16 14:03:16 EDT
I got it only once when I used the J9 VM I got around September 1st. It never
showed up again. I got it doing a Ctrl + Shift + T.
I ugrade the J9 VM when I see an update. I didn't see any update for the last
few days.
I upgrade Eclipse after each integration build and I use the jxe support.

I am using a uniprocessor machine.

I will add your command line argument in case it is happening again.
Comment 10 Kent Johnson CLA 2004-09-16 15:04:47 EDT
FYI: J9 + JXE is a relatively new combination for most eclipse developers... 
as in less than 1 month.

Prior to that most were not using J9, so we cannot really say when it first 
appeared. It may have been there since day 1.
Comment 11 Dani Megert CLA 2004-09-17 03:43:56 EDT
>Does it happen 2 mins into a run.. or if you have Eclipse up for 5 days then it 
happens..
I exit Eclipse in the evening i.e. it's running less than 1 day. When I had it
(also happened with newer J9 than Sept.1) I was after > 1 hour of working. I can
no longer work and have to exit (see comment 2).

>I'm assuming it's seen on a uniprocessor box?
Yes.


Comment 12 Markus Keller CLA 2004-09-20 11:54:44 EDT
Created attachment 14651 [details]
console output

I just got the IllegalMonitorStateException again with Eclipse I200409200800 on
a J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 j9n142-20040914 (JIT enabled)
J9VM - 20040913_1629_lHdSMR
JIT  - r7_level20040912_1800
... with a patched j9thr22.dll from Michael Brown.

Eclipse's process ID was 3292. After the exception, the eclipse process took
100% CPU.
Comment 13 Markus Keller CLA 2004-09-20 11:56:04 EDT
Created attachment 14652 [details]
stackdump
Comment 14 Markus Keller CLA 2004-09-20 11:56:54 EDT
Created attachment 14653 [details]
System events captured by DebugView
Comment 15 Markus Keller CLA 2004-09-20 13:43:26 EDT
Created attachment 14655 [details]
stackdump just before exception is thrown

Automatically created stackdump (I started with
-Xdump:heap+java:events=throw,filter=java/lang/IllegalMonitorStateException
). This came about an hour after launching the VM. That's the second one I got
with I200409200800 in a few hours. Working with the 3.0.1 Maintenance build did
not trigger one in two days. I'll try with an older vm and latest Eclipse
now...
Comment 16 Dani Megert CLA 2004-09-21 04:31:10 EDT
got it again with this setup:
java version "1.4.2"
Java(TM) 2 Runtime Environment, Standard Edition (build 2.2)
IBM J9SE VM (build 2.2, J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 j9n142-20040915
(JIT enabled)
J9VM - 20040914_1123_lHdSMR
JIT  - r7_level20040912_1800)

J9 JXE support is now enabled.

java.lang.IllegalMonitorStateException
        at java.lang.Object.notifyAll(Native Method)
        at
org.eclipse.jdt.internal.core.search.indexing.ReadWriteMonitor.exitRead(ReadWriteMonitor.java:58)
        at
org.eclipse.jdt.internal.core.search.PatternSearchJob.search(PatternSearchJob.java:119)
        at
org.eclipse.jdt.internal.core.search.PatternSearchJob.execute(PatternSearchJob.java:64)
        at
org.eclipse.jdt.internal.core.search.processing.JobManager.performConcurrentJob(JobManager.java:260)
        at
org.eclipse.jdt.core.search.SearchEngine.searchAllTypeNames(SearchEngine.java:870)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache.search(AllTypesCache.java:532)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache$TypeCacher.doSearchTypes(AllTypesCache.java:197)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache$TypeCacher.run(AllTypesCache.java:166)
Comment 17 Markus Keller CLA 2004-09-21 08:53:06 EDT
Just got it again in I20040921-0010 with an old J9 VM from July, no JXE:
IBM J9SE VM (build 2.2, J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 build
20040720_2050_lHdSMR (JIT enabled - r7_level20040720_1800))

BTW: the 100% CPU from comment 12 is most probably unrelated.
Comment 18 Kent Johnson CLA 2004-09-21 11:26:19 EDT
Another instance of a potential VM problem with IllegalMonitorStateExceptions 
is in bug 73603.
Comment 19 Nick Edgar CLA 2004-09-21 13:40:23 EDT
I just hit the case in bug 73603 after upgrading to the latest 3.1 build.
I'm using build I200409210800 + JXE support on the 20040917 VM.
Comment 20 Michael T. Brown CLA 2004-09-21 13:57:53 EDT
I've created an internal PR for this: https://bugs.ottawa.ibm.com/show_bug.cgi?
id=107454
Comment 21 Nick Edgar CLA 2004-09-21 14:34:45 EDT
I can reproduce this pretty consistently (2 for 2), just by doing a synchronize
all on a fairly typical self-hosting workspace.

Comment 22 Nick Edgar CLA 2004-09-21 16:20:57 EDT
I backed out of using the JXE support (going back to the startup.jar from the
build as well), and it still happens.

If I turn off the JIT, all seems well (albeit -very- slow).

Michael, note that the stack in bug 73603 (which is what I'm seeing) is similar
but not identical to the one here.

Is there an option for enabling the JIT overall, but disabling the optimization
that you think may be the problem here?  The option mentioned in the internal PR
disables the JIT for the specific case above, but not the case in bug 73603.
Comment 23 Nick Edgar CLA 2004-09-21 17:05:48 EDT
Just got another case, after re-enabling the JIT:

java.lang.IllegalMonitorStateException
        at java.lang.Object.notifyAll(Native Method)
        at
org.eclipse.jdt.internal.core.search.indexing.ReadWriteMonitor.exitRead(ReadWriteMonitor.java:58)
        at
org.eclipse.jdt.internal.core.search.PatternSearchJob.search(PatternSearchJob.java:119)
        at
org.eclipse.jdt.internal.core.search.PatternSearchJob.execute(PatternSearchJob.java:64)
        at
org.eclipse.jdt.internal.core.search.processing.JobManager.performConcurrentJob(JobManager.java:260)
        at
org.eclipse.jdt.core.search.SearchEngine.searchAllTypeNames(SearchEngine.java:870)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache.search(AllTypesCache.java:532)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache$TypeCacher.doSearchTypes(AllTypesCache.java:197)
        at
org.eclipse.jdt.internal.corext.util.AllTypesCache$TypeCacher.run(AllTypesCache.java:166)
Comment 24 Nick Edgar CLA 2004-09-21 17:06:15 EDT
Never mind, that's the same case as above.
Comment 25 Markus Keller CLA 2004-09-23 04:27:53 EDT
Created attachment 14732 [details]
dumps20040923-1011

Eclipse I20040921-2000, with second patched j9thr22.dll from Michael.
No vm options set besides -Xj9. Note that the thread doesn't die any more,
since the workaround from bug 73254 catches the exception.
Comment 26 Olivier Thomann CLA 2004-09-23 13:42:00 EDT
*** Bug 74794 has been marked as a duplicate of this bug. ***
Comment 27 Markus Keller CLA 2004-09-24 03:42:28 EDT
Created attachment 14760 [details]
dumps20040924-0920

This time, I ran I20040923-1635 with
-Xjit:exclude={org/eclipse/jdt/internal/core/search/indexing/ReadWriteMonitor.*}


Either it's not (only?) a jit bug, or that exclusion pattern doesn't work.
Comment 28 Nick Edgar CLA 2004-09-28 13:24:25 EDT
Just saw this too (the original stack above), on M2.
As others have mentioned, the thread no longer dies.  I just noticed it in the
error log.
I'm running with the instrumented j9thr22.dll, but I saw no output of interest
in the DebugView app.
Comment 29 Markus Keller CLA 2004-09-28 13:52:00 EDT
I didn't get the IMSE again when running with -Xjit:limitFile=C:\e\jit-limit.txt
where jit-limit.txt is (without quotes):
"- org/eclipse/jdt/internal/core/search/indexing/ReadWriteMonitor.*;
"
Comment 30 Nick Edgar CLA 2004-10-05 16:01:04 EDT
Chatting with Michael, he's wondering what other threads could be contending
with locks held by the thread that updates the AllTypesCache.

What is the best way to flush the AllTypesCache?
I can delete all the .index files from the JDT core metadata area, but that
seems a little heavy-handed.
Comment 31 Jerome Lanneluc CLA 2004-10-06 06:34:05 EDT
The AllTypeCache is just an in-memory cache. To flush it, you can either
exit/enter Eclipse. Or even simpler, you can add a new type in a test project.
Comment 32 Michael T. Brown CLA 2004-10-12 10:35:36 EDT
Just a ping/poll to see what the latest on this is. I was chatting with Markus
on Friday, and he said that he's running:

Eclipse version I20041005 with ibm-java2-ws-sdk-pj9n142-20040928a

java.fullversion=J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 j9n142-20040917 (JIT
enabled)

J9VM - 20040916_0908_lHdSMR

JIT  - r7_level20040915_1801

which is -- I believe -- the latest, and he hasn't encountered any problems in
some time. If I'm wrong on the verions, please let me know. 

The question is: are other people still seeing the problem(s) reported, and if
so, what versions (Eclipse, vm, jit etc) are you using? Has anyone been running
with the dumping options on the command line so that it'll generate a .dmp upon
an IMSE being thrown? 
Comment 33 Frederic Fusier CLA 2004-10-12 13:27:46 EDT
I got this exception (again, see bug 75321) today:
!SESSION Oct 12, 2004 18:43:42.108 ---------------------------------------------
eclipse.buildId=I200410050800
java.fullversion=J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 j9n142-20040917 (JIT
enabled)
J9VM - 20040916_0908_lHdSMR
JIT  - r7_level20040915_1801
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments: -showlocation -showversion -verify

!ENTRY org.eclipse.core.runtime 4 2 Oct 12, 2004 18:43:42.118
!MESSAGE An internal error occurred during: "Comparing tags v_510 and HEAD of
org.eclipse.jdt.core".
!STACK 0
java.lang.IllegalMonitorStateException
	at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1249)
	at java.text.DateFormat.parse(DateFormat.java:345)
	...

Is "-Xrunjdwp:onthrow=IllegalMonitorStateException" the correct dumping option
to get .dmp file when IMSE is thrown?
Comment 34 Olivier Thomann CLA 2004-10-12 16:43:28 EDT
The right option is:
-Xdump:heap+java:events=throw,filter=java/lang/IllegalMonitorStateException
Comment 35 Frederic Fusier CLA 2004-10-23 04:51:48 EDT
I got it again this morning using build I200410190941:
!ENTRY org.eclipse.ui 4 4 2004-10-22 17:05:48.217
!MESSAGE null is not a valid charset.
!SESSION 2004-10-23 10:10:24.928 -----------------------------------------------
eclipse.buildId=I200410190941
java.fullversion=J2RE 1.4.2 IBM J9 2.2 Windows XP x86-32 j9n142-20040917 (JIT
enabled)
J9VM - 20040916_0908_lHdSMR
JIT  - r7_level20040915_1801
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments: -showlocation -showversion -verify

!ENTRY org.eclipse.ui 4 4 2004-10-23 10:10:24.938
!MESSAGE null is not a valid charset.

!ENTRY org.eclipse.core.runtime 4 2 2004-10-23 10:21:55.871
!MESSAGE An internal error occurred during: "Synchronizing CVS
(/org.eclipse.jdt.core, /org.eclipse.jdt.core.tests.builder,
/org.eclipse.jdt.core.tests.compiler, /org.eclipse.jdt.core.tests.model,
/org.eclipse.jdt.core.tests.performance)".
!STACK 0
java.lang.IllegalMonitorStateException
	at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1249)
	at java.text.DateFormat.parse(DateFormat.java:345)
	at
org.eclipse.team.internal.ccvs.core.util.CVSDateFormatter.entryLineToDate(DateFormat.java)

As I had option -Xdump activated, it created lot of files (33 exactly) before I
can see this exception in the console. All these files takes around 3Go of my
disk and 810MB compressed.

I do not want to attach them to the bug. So, do I have to send all these files
or only last heapdump....dmp and javacore....txt?
Comment 36 Frederic Fusier CLA 2004-10-26 12:07:16 EDT
Michael,
As i said in previous comment 35, I succeeded to catch this exception and have
many dump files written while it happened.
If you're interested, I can send them directly to you for investigation...
Comment 37 Frederic Fusier CLA 2004-10-27 10:26:12 EDT
*** Bug 76137 has been marked as a duplicate of this bug. ***
Comment 38 Michael T. Brown CLA 2004-10-28 17:59:04 EDT
One suggestion: if you're getting this regularly (Frédéric, je vous regarde) is 
to try running this with -Xjit:noopt to reduce the work the jit does to see if 
it makes any difference. 
Comment 39 Tom Hofmann CLA 2004-11-01 06:45:39 EST
just wanted to confirm that this (bug 73603 with the SimpleDateFormat case,
actually) also happens on linux-gtk (fedora core 2, LD_ASSUME_KERNEL=2.4) with
this VM:

J9VM - 20040916_0910_lHdSMR
JIT  - r7_level20040915_1801

changing OS to 'All'.
Comment 40 Michael T. Brown CLA 2004-11-03 16:33:51 EST
This has been confirmed as a problem in the J9 VM and is being addressed. See
the internal PR for more info. 

I'll mark this bug as "invalid" 'cause it's not a problem with Eclipse proper.

Many thanks to all for the help getting information together to track this down. 
Comment 41 Michael Valenta CLA 2005-01-06 10:10:36 EST
*** Bug 82283 has been marked as a duplicate of this bug. ***