Bug 578171 - [macOS 12] JVM crash when showing Shell during menu bar browsing
Summary: [macOS 12] JVM crash when showing Shell during menu bar browsing
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.23   Edit
Hardware: Macintosh Mac OS X
: P3 normal (vote)
Target Milestone: 4.24 M1   Edit
Assignee: Alexandr Miloslavskiy CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Monterey
  Show dependency tree
 
Reported: 2022-01-11 17:00 EST by Alexandr Miloslavskiy CLA
Modified: 2022-08-23 17:17 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandr Miloslavskiy CLA 2022-01-11 17:00:16 EST
Since macOS 12, we're receiving a number of crashes related to top-level menu bar. It is currently unknown how to reproduce, but I suspect that it could be related to disabling or removing menu items.

The crash log is like:
--------
Current thread (0x00007f867480e000):  JavaThread "main" [_thread_in_native, id=259, stack(0x00007ff7bc3e0000,0x00007ff7bcbe0000)]

Stack: [0x00007ff7bc3e0000,0x00007ff7bcbe0000],  sp=0x00007ff7bcbdb2e0,  free space=8172k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [HIToolbox+0x25041]  TestWindowGroupAttributes+0x4
C  [HIToolbox+0x793e9]  _ShowHideWindows+0x4c7
C  [HIToolbox+0x975b4]  ShowMenuWindow+0x163
C  [HIToolbox+0x1b698f]  _ZL16ShowAndCacheMenuP14MenuSelectDataP8MenuDatahPK4RectP9__CFArrayh+0x6a
C  [HIToolbox+0x1b3419]  _ZL11DrawTheMenuP14MenuSelectDataP8MenuDataPP9__CFArrayhPh+0x3fe
C  [HIToolbox+0xa413a]  _ZL11MenuChangedP14MenuSelectDatahh+0x100
C  [HIToolbox+0x1b62c8]  _ZL15TrackMenuCommonR14MenuSelectDataPhP13SelectionDataP10MenuResultS5_+0x463
C  [HIToolbox+0xa3e48]  _ZL14MenuSelectCoreP8MenuData5PointdjPP13OpaqueMenuRefPt+0x18d
C  [HIToolbox+0xa3c20]  _HandleMenuSelection2+0x1cb
C  [AppKit+0x1ddc39]  _NSHandleCarbonMenuEvent+0xd7
C  [AppKit+0x1ddaa6]  _DPSEventHandledByCarbon+0x36
C  [AppKit+0x3d9ac]  -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:]+0xd77
C  [libswt-pi-cocoa-4946r10.jnilib+0x13798]  Java_org_eclipse_swt_internal_cocoa_OS_objc_1msgSendSuper__Lorg_eclipse_swt_internal_cocoa_objc_1super_2JJJJZ+0xa8
J 4998  org.eclipse.swt.internal.cocoa.OS.objc_msgSendSuper(Lorg/eclipse/swt/internal/cocoa/objc_super;JJJJZ)J (0 bytes) @ 0x000000010f6ba7b1 [0x000000010f6ba6c0+0x00000000000000f1]
J 29362 c2 org.eclipse.swt.widgets.Display.applicationProc(JJJJJJ)J (99 bytes) @ 0x0000000110216f20 [0x0000000110216c40+0x00000000000002e0]
v  ~StubRoutines::call_stub
V  [libjvm.dylib+0x3a8050]  _ZN9JavaCalls11call_helperEP9JavaValueRK12methodHandleP17JavaCallArgumentsP6Thread+0x220
V  [libjvm.dylib+0x3ea690]  _ZL17jni_invoke_staticP7JNIEnv_P9JavaValueP8_jobject11JNICallTypeP10_jmethodIDP18JNI_ArgumentPusherP6Thread+0x122
V  [libjvm.dylib+0x3ec2e6]  jni_CallStaticLongMethodV+0x12b
C  [libswt-cocoa-4946r10.jnilib+0x4534]  callback+0x4c4
C  [libswt-cocoa-4946r10.jnilib+0x19818]  fn3_6+0x48
C  [libswt-pi-cocoa-4946r10.jnilib+0xf1c4]  Java_org_eclipse_swt_internal_cocoa_OS_objc_1msgSend__JJJJJZ+0x54
J 5109  org.eclipse.swt.internal.cocoa.OS.objc_msgSend(JJJJJZ)J (0 bytes) @ 0x000000010f6c83a0 [0x000000010f6c82c0+0x00000000000000e0]
J 8943 c2 org.eclipse.swt.widgets.Display.readAndDispatch()Z (256 bytes) @ 0x000000010f6dcd68 [0x000000010f6dbf40+0x0000000000000e28]
<...>

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
J 4998  org.eclipse.swt.internal.cocoa.OS.objc_msgSendSuper(Lorg/eclipse/swt/internal/cocoa/objc_super;JJJJZ)J (0 bytes) @ 0x000000010f6ba734 [0x000000010f6ba6c0+0x0000000000000074]
J 29362 c2 org.eclipse.swt.widgets.Display.applicationProc(JJJJJJ)J (99 bytes) @ 0x0000000110216f20 [0x0000000110216c40+0x00000000000002e0]
v  ~StubRoutines::call_stub
J 5109  org.eclipse.swt.internal.cocoa.OS.objc_msgSend(JJJJJZ)J (0 bytes) @ 0x000000010f6c8323 [0x000000010f6c82c0+0x0000000000000063]
J 8943 c2 org.eclipse.swt.widgets.Display.readAndDispatch()Z (256 bytes) @ 0x000000010f6dcd68 [0x000000010f6dbf40+0x0000000000000e28]
<...>

siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x000000000000000c
--------
Comment 1 Eclipse Genie CLA 2022-02-24 22:07:20 EST
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.swt/+/191209
Comment 3 Alexandr Miloslavskiy CLA 2022-03-10 12:19:39 EST
Lakshmi, thanks for handling the patch!
Didn't expect that you will even install new OS to test it :)
Comment 4 Lakshmi P Shanmugam CLA 2022-03-14 12:13:49 EDT
Thanks Alexandr for fixing this!
Comment 5 Lakshmi P Shanmugam CLA 2022-03-14 12:19:36 EDT
(In reply to Alexandr Miloslavskiy from comment #3)
> Lakshmi, thanks for handling the patch!
> Didn't expect that you will even install new OS to test it :)

No problem, installing it was straightforward. :)
Comment 6 Eric Schlegel CLA 2022-08-21 21:07:40 EDT
Hi, this is Eric Schlegel, ericsc@apple.com. I am the engineer responsible for NSMenu. I came across this commit in eclipse while working on a binary compatibility issue for eclipse when running on macOS Ventura (menu windows are not visible when opened from the menubar).

I wanted to let you know that the NSMenuWindowManagerWindowShouldSetVisible user default was not intended for use by third-party code (in fact, it was only put in place for our own purposes during macOS development), and we will be removing support for the behavior that you are opting into here before macOS Ventura ships to customers. In general, undocumented user defaults are undocumented for a reason; we don't intend for them to be used outside of Apple. 

I suggest undoing this change in eclipse and determining if the crashes that you saw before are still happening, and if so, please file a bug with Apple as soon as possible. You can send me the feedback number at ericsc@apple.com. If I have time before Ventura ships, I will try to identify the root cause of the crash and see if there's a fix we can make in AppKit, or if I can recommend a change in eclipse. If there is not time for this before Ventura GM, then I'll continue to look into the problem for a software update release.
Comment 7 Alexandr Miloslavskiy CLA 2022-08-22 07:48:04 EDT
The crash still happens in Ventura. Easily reproducible with a snippet attached here:
https://git.eclipse.org/r/c/platform/eclipse.platform.swt/+/191209/4/tests/org.eclipse.swt.tests/ManualTests/org/eclipse/swt/tests/manual/Bug578171_macOS_JvmCrash_ShowMenuWindow.java

We already changed the workaround, so that menus work in Ventura:
https://github.com/eclipse-platform/eclipse.platform.swt/issues/256

> please file a bug with Apple as soon as possible

I have pretty terrible experience with reporting bugs to OS vendors. Chances are, I will spend effort completely in vain and no actual humans will ever look at my report. If you can ensure that bug report is actually processed, I could submit it.
Comment 8 Alexandr Miloslavskiy CLA 2022-08-22 07:50:23 EDT
> In general, undocumented user defaults are undocumented for a reason

Things go upside down when there's a nasty bug in OS itself and there's no other way to suppress it. In such cases, resorting to undocumented stuff is simply the only way, whether we like it or not.
Comment 9 Alexandr Miloslavskiy CLA 2022-08-23 17:17:46 EDT
My apologies. The crash isn't an Apple bug afterall (but there's still a bug where Menu incorrectly activates an NSWindow it shouldn't be activating, which leads to the crash).

The crash itself is triggered by calling `CancelMenuTracking()`, which, it seems, shall not be called in 64-bit code.

I have now prepared a native C++ snippet that demonstrates both problems, see https://github.com/eclipse-platform/eclipse.platform.swt/issues/337

The first bug is that when popup shell shows, it gets activated. This is an actual bug in macOS, well, according to the best of my knowledge.

The second problem is that it crashes after calling `CancelMenuTracking()`. You could say that we shouldn't be calling that, and I wouldn't object.