Bug 576099 - [Mac] JVM Crash in TreeViewer#refresh() method
Summary: [Mac] JVM Crash in TreeViewer#refresh() method
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.21   Edit
Hardware: Macintosh Mac OS X
: P3 critical with 5 votes (vote)
Target Milestone: ---   Edit
Assignee: Platform-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: needinfo, regression, vm
: 575141 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-09-20 04:30 EDT by Amit Kumar Mondal CLA
Modified: 2022-11-03 05:38 EDT (History)
12 users (show)

See Also:


Attachments
Crash Dump (139.43 KB, text/plain)
2021-09-20 04:30 EDT, Amit Kumar Mondal CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Amit Kumar Mondal CLA 2021-09-20 04:30:12 EDT
Created attachment 287167 [details]
Crash Dump

The recent version of SWT encounters a strange VM crash issue in Eclipse 2021-06 and 2021-09. The issue can be reproduced in the following manner:

1. Install the last release version of Bndtools (5.3.0-REL) from Eclipse marketplace
2. Bndtools comprises an explorer ('Bndtools explorer') which is extended from Eclipse's default 'Package Explorer'. Open this Bndtools explorer.
3. Create a dummy Bnd workspace from the File Menu
4. Create a dummy Bnd project from the File Menu
5. Open any file (e.g bnd.bnd) from the newly created project
6. Close the file
7. Collapse the project from the Bndtools explorer

Note that, this cannot be reproduced in any of the older versions of Eclipse other than the versions starting from 2021-06.

Also note that, it happens even if you open any file from any project, close the file and collapse the corresponding project in the Bndtools Explorer.

For further information, please have a look at the attached crash dump.

Affected Eclipse Version: Eclipse 2021-06, Eclipse 2021-09
Bndtools Explorer: https://github.com/bndtools/bnd/blob/5.3.0.REL/bndtools.core/src/bndtools/explorer/BndtoolsExplorer.java
Comment 1 Amit Kumar Mondal CLA 2021-09-20 04:35:22 EDT
You can also refer to this video to reproduce the problem: https://bit.ly/3tTdCRf
Comment 2 Amit Kumar Mondal CLA 2021-09-20 06:01:00 EDT
Another important point to note that the bug can only be reproduced on macOS. Tests have been performed on Windows as well but it cannot be reproduced there.
Comment 3 Niraj Modi CLA 2021-09-21 03:28:22 EDT
Hi Amit,
Are you able to reproduce this with plain Eclipse(without any external plugin) ?

Also, if you can share an SWT only test snippet to reproduce this problem at our side that will help, Thanks!
Comment 4 Amit Kumar Mondal CLA 2021-09-21 03:55:59 EDT
Hi Niraj,

Unfortunately it doesn't occur while using default 'Package Explorer' but the 'Bndtools Explorer'. I have enclosed the link to the 'Bndtools Explorer' as it extends the 'Package Explorer' and it doesn't happen in any other versions except Eclipse 2021-06 and 2021-09.

Best,
Amit
Comment 5 Victor Toni CLA 2021-09-23 13:43:04 EDT
Having the same issue on MacOS 11.5 & 11.6. Using the same configuration on Win10 as on MacOS (having an Oomph setup) gives no issues.
Same plugin, same JVM, other (older) version(s) of Eclipse also no issue. Have never Eclipse seen die like that.
Would be more than happy to provide additional information if needed, I just have to admit I really don't know anything about Eclipse internals.
Comment 6 arjan tijms CLA 2021-09-26 13:39:08 EDT
This looks similar if not identical to the following crash I get when closing tree nodes in the Project Explorer:

Process:               eclipse [44148]
Path:                  /Users/USER/*/Eclipse.app/Contents/MacOS/eclipse
Identifier:            org.eclipse.platform.ide
Version:               4.21.0 (4.21.0.I20210906-0500)
Code Type:             X86-64 (Native)
Parent Process:        ??? [1]
Responsible:           eclipse [44148]

Date/Time:             2021-09-25 20:28:42.760 +0200
OS Version:            macOS 11.6 (20G165)
Report Version:        12
Bridge OS Version:     5.5 (18P4759a)


System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       EXC_I386_GPFLT
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff204c092e __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff204ef5bd pthread_kill + 263
2   libsystem_c.dylib             	0x00007fff20444406 abort + 125
3   libjvm.dylib                  	0x000000000997e3c1 os::abort(bool, void*, void const*) + 49
4   libjvm.dylib                  	0x0000000009b61248 VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long) + 2984
5   libjvm.dylib                  	0x0000000009b60675 VMError::report_and_die(Thread*, unsigned int, unsigned char*, void*, void*, char const*, ...) + 149
6   libjvm.dylib                  	0x0000000009b612e1 VMError::report_and_die(Thread*, unsigned int, unsigned char*, void*, void*) + 33
7   libjvm.dylib                  	0x0000000009a2a4b9 JVM_handle_bsd_signal + 265
8   libsystem_platform.dylib      	0x00007fff20534d7d _sigtramp + 29
9   ???                           	0x0000000000000400 0 + 1024
10  com.apple.Foundation          	0x00007fff2131a61d -[NSConcreteMapTable objectForKey:] + 62
11  com.apple.AppKit              	0x00007fff22f1da72 -[NSOutlineView rowForItem:] + 75
12  libswt-pi-cocoa-4946r21.jnilib	0x00000000236d9a45 Java_org_eclipse_swt_internal_cocoa_OS_objc_1msgSend__JJJ + 53
13  ???                           	0x0000000015508860 0 + 357599328
14  ???                           	0x0000000018c11f78 0 + 415309688

The crashing happens intermittently, but often enough to not classify it as rare. 

Reproduced by:

1. Download Eclipse IDE for Enterprise Java and Web Developers from https://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/2021-09/R/eclipse-jee-2021-09-R-macosx-cocoa-x86_64.tar.gz
2. File -> Import -> Git -> Projects from Git -> https://github.com/eclipse-ee4j/glassfish.git
3. Start opening and closing folders from the "glassfish" project in the Project Explorer view. It may take a while, but eventually the crash will happen.
Comment 7 Amit Kumar Mondal CLA 2021-09-27 06:06:00 EDT
After digging deep into it, we found the problem to be in the TreeViewer#refresh() method. A workaround fix has been provided to Bndtools which prevents us from encountering JVM crashes while using Bndtools Explorer. However, other plugins making use of `getTreeViewer().refresh()` will most probably crash the JVM as well. 

For reference, please have a look at https://github.com/bndtools/bnd/pull/4862.

If you need, we would be very happy to provide further support on it.
Comment 8 arjan tijms CLA 2021-09-27 06:53:26 EDT
As the bug also happens in a core Eclipse plug-ins, further details would indeed be appreciated.
Comment 9 Amit Kumar Mondal CLA 2021-09-27 15:16:48 EDT
The problem occurs when from PackageExplorerPart#getTreeViewer().refresh() is called directly. Internally SWT/JFace should ensure that it gets executed in the Display thread. We expect that the call has not been scheduled to the display thread properly. That's why, after we explicitly submitted the getTreeViewer().refresh() job to the Display thread [1], the problem disappeared.

[1] - Display.getDefault().asyncExec(getTreeViewer()::refresh);

Other plugins using getTreeViewer().refresh() from PackageExplorerPart are still prone to crash the JVM as well.
Comment 10 Andrey Loskutov CLA 2021-09-27 16:02:25 EDT
(In reply to Amit Kumar Mondal from comment #7)
> After digging deep into it, we found the problem to be in the
> TreeViewer#refresh() method. A workaround fix has been provided to Bndtools
> which prevents us from encountering JVM crashes while using Bndtools
> Explorer. However, other plugins making use of `getTreeViewer().refresh()`
> will most probably crash the JVM as well. 
> 
> For reference, please have a look at
> https://github.com/bndtools/bnd/pull/4862.
> 
> If you need, we would be very happy to provide further support on it.

The point here is: in Eclipse, it should be impossible to call into SWT native code from non-UI thread, but it seem to me, that it happens here. Either this, or the call happens on a native event handler callback that doesn't expect to return into native UI code in same thread and do there some three modifications (which is what probably refresh() will do.

Can you please provide JVM crash dump or full call stack leading to the crash?
Comment 11 Amit Kumar Mondal CLA 2021-09-28 02:18:55 EDT
The crash dump has already been enclosed to the bug attachments section.
Comment 12 Niraj Modi CLA 2021-09-28 04:12:01 EDT
(In reply to Amit Kumar Mondal from comment #9)
> The problem occurs when from PackageExplorerPart#getTreeViewer().refresh()
> is called directly. Internally SWT/JFace should ensure that it gets executed
> in the Display thread. We expect that the call has not been scheduled to the
> display thread properly. That's why, after we explicitly submitted the
> getTreeViewer().refresh() job to the Display thread [1], the problem
> disappeared.
> 
> [1] - Display.getDefault().asyncExec(getTreeViewer()::refresh);
> 
> Other plugins using getTreeViewer().refresh() from PackageExplorerPart are
> still prone to crash the JVM as well.

Moving to Platfrom-UI to evaluate possible fix here.
Comment 13 arjan tijms CLA 2021-10-13 12:14:39 EDT
Any updates here?
Comment 14 arjan tijms CLA 2021-10-19 15:34:38 EDT
Similar of the same rapport here: https://bugs.eclipse.org/bugs/show_bug.cgi?id=575141
Comment 15 Lakshmi P Shanmugam CLA 2021-10-27 05:19:58 EDT
*** Bug 575141 has been marked as a duplicate of this bug. ***
Comment 16 Lakshmi P Shanmugam CLA 2021-10-27 05:32:17 EDT
From the stacktrace, NSOutlineView.rowForItem is crashing and looks like this call in drawInteriorWithFrame_inView called with an invalid value. AFAICS there were no changes in this area from SWT POV. Trying to reproduce it locally to investigate further.

Can you please specify the Java version?
Comment 17 Thomas Wolf CLA 2021-10-27 05:54:18 EDT
(In reply to Lakshmi P Shanmugam from comment #16)
> Can you please specify the Java version?

The crash dump attached says Java 15. I got bug 575141 with Amazon Corretto 11.
Comment 18 arjan tijms CLA 2021-10-31 11:42:09 EDT
For me the bug happened with JDK 16:

java.runtime.name=OpenJDK Runtime Environment
java.runtime.version=16.0.2+7-67
java.specification.name=Java Platform API Specification
java.specification.vendor=Oracle Corporation
java.specification.version=16
java.vendor=Oracle Corporation
java.vendor.url=https://java.oracle.com/
java.vendor.url.bug=https://bugreport.java.com/bugreport/
java.version=16.0.2
java.version.date=2021-07-20
java.vm.compressedOopsMode=32-bit
java.vm.info=mixed mode
java.vm.name=OpenJDK 64-Bit Server VM
java.vm.specification.name=Java Virtual Machine Specification
java.vm.specification.vendor=Oracle Corporation
java.vm.specification.version=16
java.vm.vendor=Oracle Corporation
java.vm.version=16.0.2+7-67
Comment 19 Adam Gent CLA 2022-03-31 14:44:37 EDT
FWIW it still happens for me on Mojave w/ JDK 18.

Anytime I go about three levels deep in the call hierarchy view.
Comment 20 David Durand CLA 2022-06-20 20:55:50 EDT
This bug happens to me all the time (really only in the call hierarchy, when expanding an item). It happens on Mojave and Catalina, and is still happening with Eclipse 4.24 (2022-06 release).

The fact that the original reporter was able to fix their application with a 3-line fix seems like something to investigate. If I could find the source code, I would at least examine the update() method of the call hierarchy model to see if the workaround might be applied there.

I wish that I could help, but it's very difficult to find out how to get a build setup going to try to hack at this bug.

I have tried many different JVMs and variations of plugins, (versions 11-1, openjdk, graalvm, justj) but it never seems to have gone away, though it did seem to quiet down until I updated to 2022-06.

I don't know how long I can continue flinching in fear whenever I have to look at function callers, with the chance of a disruptive crash.
Comment 21 Lakshmi P Shanmugam CLA 2022-06-21 02:09:20 EDT
(In reply to David Durand from comment #20)

Are you able to reproduce consistently? Can you provide any steps to reproduce?
I've been unable to reproduce this, expanding items in the Call hierarchy view in the Eclipse SDK seems to work fine for me.
Comment 22 David Durand CLA 2022-07-07 15:44:26 EDT
(In reply to Lakshmi P Shanmugam from comment #21)
> (In reply to David Durand from comment #20)
> 
> Are you able to reproduce consistently? Can you provide any steps to
> reproduce?
> I've been unable to reproduce this, expanding items in the Call hierarchy
> view in the Eclipse SDK seems to work fine for me.

It happens to me freq
Comment 23 David Durand CLA 2022-07-07 16:15:28 EDT
(In reply to Lakshmi P Shanmugam from comment #21)
> (In reply to David Durand from comment #20)
> 
> Are you able to reproduce consistently? Can you provide any steps to
> reproduce?
> I've been unable to reproduce this, expanding items in the Call hierarchy
> view in the Eclipse SDK seems to work fine for me.

Trying again, as the browser seems to have prematurely posted. It happens regularly to me with various VMs (I went through a period of trying different versions and builds of Java to see if that helped). I am running older MacOS versions, but I see it on both Mojave and Catalina. I would be happy to add more crash dumps -- though I have not saved most of them. I just crashed it again, a few minutes ago, trying to check if it's still happening. It worked for the first hierarchy, but then I opened it on different call, and the first expansion of a caller crashed it again. It's always on expansion of a second level tab, or deeper. It would happen to me a lot more often in regular work, but I now often re-open the view on a deeper entry rather than taking the chance of losing my context by expanding an entry. I've never seen it happen on immediate display.

Like most Eclipse installations (I imagine) I have a bunch of different plugins, but this problem seems to be related purely to the Java call hierarchy. I almost always use it to find callers of the current item specified. It's a medium-large project, around 120kloc of Java code, and this may matter as a larger project might change the timing, making the posting of the view update more likely to happen after a delay, and perhaps in a different thread. I've seen the crash on relatively virgin installs of the Java tooling, before importing plugins from my old installs.

It's a pain to try to gauge the frequency of something like this, but I'd say that one expansion in 10 will cause a crash -- It's infrequent enough that will I hold my breath and expand an item now and then, but I save all my work first.

If configuration or preferences or other persistent state in the workbench would help, I'm happy to upload anything that would help.
Comment 24 Victor Toni CLA 2022-09-16 13:14:08 EDT
(In reply to David Durand from comment #20)
> This bug happens to me all the time (really only in the call hierarchy, when
> expanding an item). It happens on Mojave and Catalina, and is still
> happening with Eclipse 4.24 (2022-06 release).
> 
> The fact that the original reporter was able to fix their application with a
> 3-line fix seems like something to investigate. If I could find the source
> code, I would at least examine the update() method of the call hierarchy
> model to see if the workaround might be applied there.

The change doesn't aktually fix the problem, it just makes the fix less likely (although not for me, since the less likely is based on changed timing, since I like to navigate the tree using cursor keys I can make it break very fast...)
The actual change is here: https://github.com/bndtools/bnd/commit/a69afa66357b5c54fa437a6c28271c6159daaf77

Having tested 2022-09 I can confirm the bug is still there and easily reproducible (at least with bndtools installed).

The steps to reproduce from the initial bug description are still valid for newer versions of Eclipse and the plugin.
Comment 25 Peter Kriens CLA 2022-11-03 05:38:03 EDT
I found a consistent short sequence that core dumps. I can run this in the debugger to quite shortly before the crash happens.

Sequence

* Open bndtools explorer
* Create Java project
* Open VM entry in project
* Open a jar, then a package, then a class so editor opens
* close VM entry in project

I then nettled down the problem to a call to the Package Explorer's TreeViewer refresh() method. Note that this refresh was made on the main thread. When I removed the call to refresh, there was no longer a VM crash.

The refresh() was called rather indiscriminately. It was only necessary to call it when the filter criteria changed. In the steps towards the crash, these criteria were not affected. To fix MY problem, I just no longer call refresh
when the tree item is closed. See https://github.com/bndtools/bnd/pull/5411

However, there is of course a bug lurking in the TreeViewer ...

If there is an Eclipse developer that can do a debug session over Zoom with me then I think we can find the cause of this problem.