Bug 567881 - [GTK][webkit] Hang on Ubuntu in gtk.OS.g_main_context_iteration after Webkit2AsyncToSync.execAsyncAndWaitForReturn
Summary: [GTK][webkit] Hang on Ubuntu in gtk.OS.g_main_context_iteration after Webkit2...
Status: REOPENED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.18   Edit
Hardware: PC Linux
: P3 normal with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: needinfo
Depends on:
Blocks:
 
Reported: 2020-10-14 13:48 EDT by Gunnar Wagenknecht CLA
Modified: 2021-06-08 12:28 EDT (History)
5 users (show)

See Also:


Attachments
full thread dump (203.96 KB, text/plain)
2020-10-14 13:48 EDT, Gunnar Wagenknecht CLA
no flags Details
JVM Crash Log (563.21 KB, text/plain)
2020-11-01 01:26 EST, Gunnar Wagenknecht CLA
no flags Details
screenshot when freezing (1.98 MB, image/png)
2020-12-04 05:53 EST, Gunnar Wagenknecht CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Gunnar Wagenknecht CLA 2020-10-14 13:48:55 EDT
Created attachment 284453 [details]
full thread dump

Eclipse 4.17

In this particular case the lock up happens when invoking code completion.





"main" #1 prio=6 os_prio=0 cpu=182585.70ms elapsed=4719.93s tid=0x00007f115001a800 nid=0x349b runnable  [0x00007f11569f2000]
   java.lang.Thread.State: RUNNABLE
	at org.eclipse.swt.internal.gtk.OS.g_main_context_iteration(Native Method)
	at org.eclipse.swt.browser.WebKit$Webkit2AsyncToSync.execAsyncAndWaitForReturn(WebKit.java:1416)
	at org.eclipse.swt.browser.WebKit$Webkit2AsyncToSync.runjavascript(WebKit.java:1185)
	at org.eclipse.swt.browser.WebKit$Webkit2AsyncToSync.evaluate(WebKit.java:1133)
	at org.eclipse.swt.browser.WebKit.evaluate(WebKit.java:1451)
	at org.eclipse.swt.browser.WebKit.close(WebKit.java:958)
	at org.eclipse.swt.browser.WebKit.onDispose(WebKit.java:1944)
	at org.eclipse.swt.browser.WebKit.lambda$4(WebKit.java:864)
	at org.eclipse.swt.browser.WebKit$$Lambda$1222/0x000000080110f040.handleEvent(Unknown Source)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5745)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1427)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1453)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1432)
	at org.eclipse.swt.widgets.Widget.release(Widget.java:1244)
	at org.eclipse.swt.widgets.Control.release(Control.java:4704)
	at org.eclipse.swt.widgets.Composite.releaseChildren(Composite.java:1505)
	at org.eclipse.swt.widgets.Widget.release(Widget.java:1247)
	at org.eclipse.swt.widgets.Control.release(Control.java:4704)
	at org.eclipse.swt.widgets.Composite.releaseChildren(Composite.java:1505)
	at org.eclipse.swt.widgets.Canvas.releaseChildren(Canvas.java:279)
	at org.eclipse.swt.widgets.Decorations.releaseChildren(Decorations.java:486)
	at org.eclipse.swt.widgets.Shell.releaseChildren(Shell.java:3249)
	at org.eclipse.swt.widgets.Widget.release(Widget.java:1247)
	at org.eclipse.swt.widgets.Control.release(Control.java:4704)
	at org.eclipse.swt.widgets.Widget.dispose(Widget.java:529)
	at org.eclipse.swt.widgets.Shell.dispose(Shell.java:3170)
	at org.eclipse.jface.text.AbstractInformationControl.dispose(AbstractInformationControl.java:500)
	at org.eclipse.jface.text.AbstractInformationControlManager.disposeInformationControl(AbstractInformationControlManager.java:1278)
	at org.eclipse.jface.text.contentassist.AdditionalInfoController.disposeInformationControl(AdditionalInfoController.java:498)
	at org.eclipse.jface.text.AbstractInformationControlManager.handleSubjectControlDisposed(AbstractInformationControlManager.java:647)
	at org.eclipse.jface.text.AbstractInformationControlManager.lambda$0(AbstractInformationControlManager.java:682)
	at org.eclipse.jface.text.AbstractInformationControlManager$$Lambda$1051/0x0000000800f5e040.widgetDisposed(Unknown Source)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:127)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5745)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1427)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1453)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1432)
	at org.eclipse.swt.widgets.Widget.release(Widget.java:1244)
	at org.eclipse.swt.widgets.Control.release(Control.java:4704)
	at org.eclipse.swt.widgets.Widget.dispose(Widget.java:529)
	at org.eclipse.swt.widgets.Shell.dispose(Shell.java:3170)
	at org.eclipse.jface.text.contentassist.CompletionProposalPopup.hide(CompletionProposalPopup.java:1089)
	at org.eclipse.jface.text.contentassist.AsyncCompletionProposalPopup.hide(AsyncCompletionProposalPopup.java:331)
	at org.eclipse.jface.text.contentassist.CompletionProposalPopup.insertSelectedProposalWithMask(CompletionProposalPopup.java:942)
	at org.eclipse.jface.text.contentassist.CompletionProposalPopup.verifyKey(CompletionProposalPopup.java:1369)
	at org.eclipse.jface.text.contentassist.ContentAssistant$InternalListener.verifyKey(ContentAssistant.java:809)
	at org.eclipse.jface.text.TextViewer$VerifyKeyListenersManager.verifyKey(TextViewer.java:481)
	at org.eclipse.swt.custom.StyledTextListener.handleEvent(StyledTextListener.java:70)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5745)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1427)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1453)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1436)
	at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1225)
	at org.eclipse.swt.custom.StyledText.handleKeyDown(StyledText.java:6099)
	at org.eclipse.swt.custom.StyledText.lambda$1(StyledText.java:5793)
	at org.eclipse.swt.custom.StyledText$$Lambda$144/0x0000000800294c40.handleEvent(Unknown Source)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:5745)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1427)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1453)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1436)
	at org.eclipse.swt.widgets.Widget.sendKeyEvent(Widget.java:1475)
	at org.eclipse.swt.widgets.Widget.gtk_key_press_event(Widget.java:838)
	at org.eclipse.swt.widgets.Control.gtk_key_press_event(Control.java:4021)
	at org.eclipse.swt.widgets.Composite.gtk_key_press_event(Composite.java:850)
	at org.eclipse.swt.widgets.Widget.windowProc(Widget.java:2326)
	at org.eclipse.swt.widgets.Control.windowProc(Control.java:6795)
	at org.eclipse.swt.widgets.Display.windowProc(Display.java:5979)
	at org.eclipse.swt.internal.gtk.GTK.gtk_main_do_event(Native Method)
	at org.eclipse.swt.widgets.Display.eventProc(Display.java:1517)
	at org.eclipse.swt.internal.gtk.OS.g_main_context_iteration(Native Method)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4505)
	at org.eclipse.jface.internal.text.html.BrowserInformationControl.setVisible(BrowserInformationControl.java:347)
	at org.eclipse.jface.text.AbstractInformationControlManager.showInformationControl(AbstractInformationControlManager.java:1240)
	at org.eclipse.jface.text.AbstractInformationControlManager.internalShowInformationControl(AbstractInformationControlManager.java:1191)
	at org.eclipse.jface.text.AbstractInformationControlManager.presentInformation(AbstractInformationControlManager.java:1120)
	at org.eclipse.jface.text.AbstractInformationControlManager.setInformation(AbstractInformationControlManager.java:431)
	at org.eclipse.jface.text.contentassist.AdditionalInfoController.computeInformation(AdditionalInfoController.java:548)
	at org.eclipse.jface.text.AbstractInformationControlManager.doShowInformation(AbstractInformationControlManager.java:1101)
	at org.eclipse.jface.text.AbstractInformationControlManager.showInformation(AbstractInformationControlManager.java:1091)
	at org.eclipse.jface.text.contentassist.AdditionalInfoController.showInformation(AdditionalInfoController.java:530)
	at org.eclipse.jface.text.contentassist.AdditionalInfoController$1.showInformation(AdditionalInfoController.java:477)
	at org.eclipse.jface.text.contentassist.AdditionalInfoController$Timer.lambda$1(AdditionalInfoController.java:362)
	- locked <0x00000003bfe8d350> (a org.eclipse.jface.text.contentassist.AdditionalInfoController$1)
	at org.eclipse.jface.text.contentassist.AdditionalInfoController$Timer$$Lambda$1388/0x0000000801185c40.run(Unknown Source)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:40)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:185)
	- locked <0x00000003b7800470> (a org.eclipse.swt.widgets.RunnableLock)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4988)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:4510)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1157)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:644)
	at org.eclipse.ui.internal.Workbench$$Lambda$193/0x00000008002cbc40.run(Unknown Source)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:551)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:153)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:150)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.6.0.101/Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@11.0.6.0.101/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.6.0.101/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@11.0.6.0.101/Method.java:566)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:657)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:594)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1438)
Comment 1 Gunnar Wagenknecht CLA 2020-10-14 13:49:39 EDT
What is the state of GTK and browser support?
Is WebKit the recommended browser?
Should we try a different one?
Comment 2 Alexander Kurtakov CLA 2020-10-14 14:19:56 EDT
(In reply to Gunnar Wagenknecht from comment #1)
> What is the state of GTK and browser support?

I'm not aware of any blockers.

> Is WebKit the recommended browser?

Yes it is. 

> Should we try a different one?

This is the single browser engine available on every linux distribution. The other one may be firefox but it is no longer suitable for embedding. If you are aware of more widely available and suitable for embedding engine I'm all ears.
Comment 3 Alexander Kurtakov CLA 2020-10-14 14:20:28 EDT
Which gtk version is that? Which distribution? X11 or wayland?
Comment 4 Andrey Loskutov CLA 2020-10-14 14:40:06 EDT
See also bug 546579, bug 548230, bug 564911 and bug 566864. Looks like an older problem with disposal of webkit browser.

All seem to hang after Webkit2AsyncToSync.execAsyncAndWaitForReturn.

Would be nice to have steps to reproduce and exact versions of GTK3 + webkit libraries installed.
Comment 5 Gunnar Wagenknecht CLA 2020-10-14 15:37:35 EDT
(In reply to Alexander Kurtakov from comment #3)
> Which gtk version is that? Which distribution? X11 or wayland?

Ubuntu 18.04
gtk 3.22.30-1ubuntu4

(In reply to Alexander Kurtakov from comment #2)
> > Should we try a different one?
> 
> This is the single browser engine available on every linux distribution. The
> other one may be firefox but it is no longer suitable for embedding. If you
> are aware of more widely available and suitable for embedding engine I'm all
> ears.

What about: -Dorg.eclipse.swt.browser.DefaultType=chromium?
Comment 6 Gunnar Wagenknecht CLA 2020-10-14 15:38:37 EDT
(In reply to Gunnar Wagenknecht from comment #5)
> What about: -Dorg.eclipse.swt.browser.DefaultType=chromium?


Fails with
Missing CEF binaries for Chromium Browser. Extract CEF binaries to /home/.../.swt/lib/linux/x86_64/chromium-3071 (java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons: 
	Can't load library: /home/.../.swt/lib/linux/x86_64/chromium-3071/libcef.so
	Can't load library: /home/.../.swt/lib/linux/x86_64//home/.../.swt/lib/linux/x86_64/chromium-3071/libcef.so
)
Comment 7 Alexander Kurtakov CLA 2020-10-14 15:48:25 EDT
(In reply to Gunnar Wagenknecht from comment #5)
> (In reply to Alexander Kurtakov from comment #3)
> > Which gtk version is that? Which distribution? X11 or wayland?
> 
> Ubuntu 18.04
> gtk 3.22.30-1ubuntu4
> 
> (In reply to Alexander Kurtakov from comment #2)
> > > Should we try a different one?
> > 
> > This is the single browser engine available on every linux distribution. The
> > other one may be firefox but it is no longer suitable for embedding. If you
> > are aware of more widely available and suitable for embedding engine I'm all
> > ears.
> 
> What about: -Dorg.eclipse.swt.browser.DefaultType=chromium?

This is not an alternative yet. It's ancient chromium, needs additional installaion, doesn't work under wayland.
Comment 8 Gunnar Wagenknecht CLA 2020-10-14 15:52:34 EDT
(In reply to Andrey Loskutov from comment #4)
> Would be nice to have steps to reproduce 

A kingdom for those! It happens on *some* Code Assists but not all. Sigh.
Comment 9 Gunnar Wagenknecht CLA 2020-10-14 15:55:17 EDT
(In reply to Alexander Kurtakov from comment #7)
> > What about: -Dorg.eclipse.swt.browser.DefaultType=chromium?
> 
> This is not an alternative yet. It's ancient chromium, needs additional
> installaion, doesn't work under wayland.

I'm on a Mac. I can see the Chromium fragments there.

I also found them in Git:
https://git.eclipse.org/c/platform/eclipse.platform.swt.binaries.git/tree/bundles/org.eclipse.swt.browser.chromium.gtk.linux.x86_64

Are you saying they are not shipped/distributed on Linux GTK?
Is there a way to try them? It's ContentAssist. I don't need a fancy browser widget. Just a thing that works. :)
Comment 10 Gunnar Wagenknecht CLA 2020-10-14 15:57:32 EDT
(In reply to Gunnar Wagenknecht from comment #9)
> Are you saying they are not shipped/distributed on Linux GTK?

I also found it in the FAQ:
https://www.eclipse.org/swt/faq.php#browserlinux

So it should work. Are they any other steps necessary other than setting the property?
Comment 11 Alexander Kurtakov CLA 2020-10-14 17:15:04 EDT
(In reply to Gunnar Wagenknecht from comment #10)
> (In reply to Gunnar Wagenknecht from comment #9)
> > Are you saying they are not shipped/distributed on Linux GTK?
> 
> I also found it in the FAQ:
> https://www.eclipse.org/swt/faq.php#browserlinux
> 
> So it should work. Are they any other steps necessary other than setting the
> property?

See bug 565507 and its friends. It never worked for me (due to being on wayland and denying playing with additional configs). Either way, this bug is not about chromium it has a number of others to be discussed.
Can you tell on which completions it happens? There might be smth crashing webkit in them. 
Btw, which version of webkitgtk? You can see it in configuration tab in About dialog.
Also is it local or some vnc(like) connection? Making sure it's not a dup of bug 567311
Comment 12 Gunnar Wagenknecht CLA 2020-10-15 01:58:29 EDT
(In reply to Alexander Kurtakov from comment #11)
> Either way, this bug
> is not about chromium it has a number of others to be discussed.

+1

> Can you tell on which completions it happens? There might be smth crashing
> webkit in them. 

No. We haven't found a pattern yet. It seems random, i.e. not reproducable.

> Btw, which version of webkitgtk? You can see it in configuration tab in
> About dialog.

I'll get the info (I am just the communication bridge, sorry).

> Also is it local or some vnc(like) connection? Making sure it's not a dup of
> bug 567311

It's definitely happening local. No VNC or the like.
Comment 13 Gunnar Wagenknecht CLA 2020-10-20 06:43:35 EDT
Alexander, is there a way to debug the GTK/WebKit combination? Any .options to turn on? Any other way to unveil more debugging output for diagnosis?
Comment 14 Alexander Kurtakov CLA 2020-10-20 06:46:58 EDT
(In reply to Gunnar Wagenknecht from comment #13)
> Alexander, is there a way to debug the GTK/WebKit combination? Any .options
> to turn on? Any other way to unveil more debugging output for diagnosis?

Eric, can you give some hints here? I know you moved on but you're the last one that touched this area.
Comment 15 Gunnar Wagenknecht CLA 2020-10-20 13:36:31 EDT
(In reply to Gunnar Wagenknecht from comment #12)
> (In reply to Alexander Kurtakov from comment #11)
> > Btw, which version of webkitgtk? You can see it in configuration tab in
> > About dialog.
> 
> I'll get the info (I am just the communication bridge, sorry).

Here is data from one of the Ubuntu 18 systems:

org.eclipse.swt.internal.deviceZoom=100
org.eclipse.swt.internal.gdk.backend=x11
org.eclipse.swt.internal.gtk.theme=Ambiance
org.eclipse.swt.internal.gtk.version=3.22.30
org.eclipse.swt.internal.webkitgtk.version=2.28.4
Comment 16 Gunnar Wagenknecht CLA 2020-10-21 02:03:21 EDT
(In reply to Gunnar Wagenknecht from comment #15)
> Here is data from one of the Ubuntu 18 systems:
> 
> org.eclipse.swt.internal.deviceZoom=100
> org.eclipse.swt.internal.gdk.backend=x11
> org.eclipse.swt.internal.gtk.theme=Ambiance
> org.eclipse.swt.internal.gtk.version=3.22.30
> org.eclipse.swt.internal.webkitgtk.version=2.28.4

So every Ubuntu 18 user is affected (which is roughly ~150 users). They all show the same version for GTK and WebkitGTK as shown above.

The problem is intermittent. It usually happens after 10-20 minutes of using Eclipse 4.17.

I'm able to push out test builds of SWT quickly. Thus, anything would be very helpful.

Is it possible to add a timeout to Webkit2AsyncToSync.execAsyncAndWaitForReturn?
Comment 17 Gunnar Wagenknecht CLA 2020-10-22 03:46:18 EDT
Some interesting news: Users are reporting that 4.14 is not showing the problem. Looks like something regressed in SWT?
Comment 18 Eric Williams CLA 2020-10-22 08:02:39 EDT
Can you provide a code snippet that reproduces bug? Or a class where, upon invoking the code assist, the bug happens?

What about error messages, any native error messages printed in the console before/after the hangup?
Comment 19 Eric Williams CLA 2020-10-22 08:22:51 EDT
Additionally, could you try on a newer Ubuntu (20.04) to see if maybe it's a bug in an older WebKit/GTK?
Comment 20 Gunnar Wagenknecht CLA 2020-10-22 09:52:43 EDT
(In reply to Eric Williams from comment #18)
> Can you provide a code snippet that reproduces bug? Or a class where, upon
> invoking the code assist, the bug happens?

Nope, it's random.


Here is the native callstack I got:

(gdb) bt #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1 0x00007f270fb26dac in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #2 0x00007f2750df3069 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0 #3 0x00007f270fae1285 in g_main_context_dispatch () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #4 0x00007f270fae1650 in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #5 0x00007f270fae16dc in g_main_context_iteration () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #6 0x00007f26b2b91ce7 in Java_org_eclipse_swt_internal_gtk_OS_g_1main_1context_1iteration () from /home/btoal/blt/extra-tools/experimental/sfdcEclipse/configuration/org.eclipse.osgi/222/0/.cp/libswt-pi3-gtk-4936r26.so #7 0x00007f27a95a8f71 in ?? () #8 0x00007f27bfb47e90 in ?? () #9 0x00007f2634326187 in ?? () #10 0x00007f27bfb47ef8 in ?? () #11 0x00007f2634328068 in ?? () #12 0x0000000000000000 in ?? ()

> What about error messages, any native error messages printed in the console
> before/after the hangup?

Nothing printed. Assuming this code is related to some locking and async processing, is it possible to get an SWT version with some additional debug output when calling/working with WebkitGTK?
Comment 21 Eric Williams CLA 2020-10-22 11:03:38 EDT
(In reply to Gunnar Wagenknecht from comment #20)
> (In reply to Eric Williams from comment #18)
> > Can you provide a code snippet that reproduces bug? Or a class where, upon
> > invoking the code assist, the bug happens?
> 
> Nope, it's random.
> 
> 
> Here is the native callstack I got:
> 
> (gdb) bt #0 syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38 #1
> 0x00007f270fb26dac in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
> #2 0x00007f2750df3069 in ?? () from /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
> #3 0x00007f270fae1285 in g_main_context_dispatch () from
> /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #4 0x00007f270fae1650 in ?? ()
> from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 #5 0x00007f270fae16dc in
> g_main_context_iteration () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
> #6 0x00007f26b2b91ce7 in
> Java_org_eclipse_swt_internal_gtk_OS_g_1main_1context_1iteration () from
> /home/btoal/blt/extra-tools/experimental/sfdcEclipse/configuration/org.
> eclipse.osgi/222/0/.cp/libswt-pi3-gtk-4936r26.so #7 0x00007f27a95a8f71 in ??
> () #8 0x00007f27bfb47e90 in ?? () #9 0x00007f2634326187 in ?? () #10
> 0x00007f27bfb47ef8 in ?? () #11 0x00007f2634328068 in ?? () #12
> 0x0000000000000000 in ?? ()
> 
> > What about error messages, any native error messages printed in the console
> > before/after the hangup?
> 
> Nothing printed. Assuming this code is related to some locking and async
> processing, is it possible to get an SWT version with some additional debug
> output when calling/working with WebkitGTK?

g_main_context_iteration() is just the main loop, so a lockup there doesn't show anything useful unfortunately.

I'll see if I can reproduce the bug this afternoon. Would you be able to try on a newer version of Ubuntu?
Comment 22 Gunnar Wagenknecht CLA 2020-10-22 12:25:41 EDT
(In reply to Eric Williams from comment #21)
> I'll see if I can reproduce the bug this afternoon. Would you be able to try
> on a newer version of Ubuntu?

I'm working with our Linux support team to get a backport of the latest version of webkitgtk for Ubuntu 18. But switching to Ubuntu 20 is unfortuntely not an option at this time.
Comment 23 Eric Williams CLA 2020-10-22 15:14:25 EDT
I have been unable to reproduce the bug on Fedora 32, X11, and Eclipse 4.16 and 4.17. I had a spare laptop floating around that I installed Ubuntu 18.04 on, and I was unable to reproduce the issue there as well (Eclipse 4.16 and 4.17).

I tried a couple of things:
- multiple code assists in a short period of time
- leave Eclipse running for 1 hour or more, and then coming back to it to do more code assists
- different types of code assists for different classes, functions vs. fields, etc.

I did notice one interesting thing. In the org.eclipse.swt.tests project there is a test suite "Test_Memory_Leak". It, in essence, repeatedly opens Browser widgets and then disposes them to check for memory leaks.

On Fedora, this test will run for 5+ minutes, hitting over 7000 instances opened/closed, and will continue onwards without issue. On Ubuntu this runs for about 2500 iterations, and then SWT crashes with a warning message about "too many files opened". It could be related.

My recommendation: see if you can't get the issue to reproduce using some of the Browser tests available:
- Test_Memory_Leak
- BrowserExample in org.eclipse.swt.examples
- AllBrowserTests in org.eclipse.swt.tests
Comment 24 Andrey Loskutov CLA 2020-10-22 15:46:51 EDT
(In reply to Eric Williams from comment #23)
> I did notice one interesting thing. In the org.eclipse.swt.tests project
> there is a test suite "Test_Memory_Leak". It, in essence, repeatedly opens
> Browser widgets and then disposes them to check for memory leaks.
> 
> On Fedora, this test will run for 5+ minutes, hitting over 7000 instances
> opened/closed, and will continue onwards without issue. On Ubuntu this runs
> for about 2500 iterations, and then SWT crashes with a warning message about
> "too many files opened". It could be related.

This reminds me on patch for Bug 564474 that we prepared but didn't merged yet, see https://git.eclipse.org/r/c/platform/eclipse.platform.swt/+/165360/

Simeon, do you remember why we didn't merged this? Could it be we've just forgotten about it? Or was it just because we had no idea about possible side effects? Eric, any comments from your side?
Comment 25 Simeon Andreev CLA 2020-10-22 15:51:56 EDT
> Simeon, do you remember why we didn't merged this? Could it be we've just
> forgotten about it? Or was it just because we had no idea about possible
> side effects? Eric, any comments from your side?

We updated our WebKit2 installation; we didn't invest more time afterwards. I assume the change itself is fine.
Comment 26 Andrey Loskutov CLA 2020-10-22 15:58:24 EDT
(In reply to Simeon Andreev from comment #25)
> We updated our WebKit2 installation; we didn't invest more time afterwards.

A, right, but I also see now that Gunnar is already on "modern" 2.28.4 version, so patch is unrelated for him.
Comment 27 Gunnar Wagenknecht CLA 2020-10-28 12:21:07 EDT
One of our users seeing this in JavaScript content assist too (it's not just Java). He happened to observe the following printed in the console:

QMetaObject::invokeMethod: No such method Gui::PendingChangeDialog::validChange(Obj::Object*)
Candidates are:
    validChange()
Comment 28 Alexander Kurtakov CLA 2020-10-28 12:23:47 EDT
(In reply to Gunnar Wagenknecht from comment #27)
> One of our users seeing this in JavaScript content assist too (it's not just
> Java). He happened to observe the following printed in the console:
> 
> QMetaObject::invokeMethod: No such method
> Gui::PendingChangeDialog::validChange(Obj::Object*)
> Candidates are:
>     validChange()

Hmm, QMetaObject is QT object. The only way I can think of QT being involved in webkitgtk/swt is if one uses some QT based Gtk theme. I recommend trying to reproduce using the default Gtk theme (Adwaita) and if it doesn't happen with it I would claim it's theme issue.
Comment 29 Gunnar Wagenknecht CLA 2020-10-28 12:26:02 EDT
(In reply to Gunnar Wagenknecht from comment #27)
> One of our users seeing this in JavaScript content assist too (it's not just
> Java). He happened to observe the following printed in the console:
> 
> QMetaObject::invokeMethod: No such method
> Gui::PendingChangeDialog::validChange(Obj::Object*)
> Candidates are:
>     validChange()

Please ignore. It might be noise. "Gui::PendingChangeDialog::validChange" seems to come from Perforce p4v tool. Likely unrelated to Eclipse.
Comment 30 Gunnar Wagenknecht CLA 2020-11-01 01:26:02 EST
Created attachment 284622 [details]
JVM Crash Log

I don't know yet if it is related. However, some of our users are experiencing a full crash while others are experiencing a hang. Same situation.
Comment 31 Simeon Andreev CLA 2020-11-01 02:06:50 EST
Possibly unrelated, we've already had 2 different crashes at the same line.

1. Due to parallel GC (doesn't seem to be the case in the attached JVM crash log): https://bugzilla.redhat.com/show_bug.cgi?id=1828845
2. Due to calling jstack (I assume the JVM was hanging and someone called jstack?): https://bugzilla.redhat.com/show_bug.cgi?id=1832121
Comment 32 Gunnar Wagenknecht CLA 2020-11-01 10:10:36 EST
(In reply to Simeon Andreev from comment #31)
> 2. Due to calling jstack (I assume the JVM was hanging and someone called
> jstack?): https://bugzilla.redhat.com/show_bug.cgi?id=1832121

Highly probably.
Comment 33 Gunnar Wagenknecht CLA 2020-12-03 14:27:44 EST
(In reply to Andrey Loskutov from comment #26)
> (In reply to Simeon Andreev from comment #25)
> > We updated our WebKit2 installation; we didn't invest more time afterwards.
> 
> A, right, but I also see now that Gunnar is already on "modern" 2.28.4
> version, so patch is unrelated for him.

The latest version is 2.30.3. This has made it to Ubuntu 18.04 due to a security issue. However, the lock still occurs there.
Comment 34 Gunnar Wagenknecht CLA 2020-12-04 05:53:50 EST
Created attachment 284961 [details]
screenshot when freezing

(In reply to Eric Williams from comment #23)
> I tried a couple of things:
> - multiple code assists in a short period of time
> - leave Eclipse running for 1 hour or more, and then coming back to it to do
> more code assists
> - different types of code assists for different classes, functions vs.
> fields, etc.

I believe it has something to do with the number of results and/or a specific result. Sometimes Eclipse freezes while the user is "browsing" through the content assist. This could indicate it being related to the doc tooltip. But most reports indicate Eclipse freezes when the user actually picks an item and hits return, which is related to closing the content assist. When it freezes, the content assist is still visible.



 
> My recommendation: see if you can't get the issue to reproduce using some of
> the Browser tests available:
> - Test_Memory_Leak


I see this error when running the test:
> Failed to launch /usr/lib/x86_64-linux-gnu/webkit2gtk-4.0/WebKitPluginProcess: Failed to execute child process “/usr/lib/x86_64-linux-gnu/webkit2gtk-4.0/WebKitPluginProcess” (No such file or directory)

Is this an issue? I see the counter and an empty SWT shell (window). No browser widget is opening/closing.
Comment 35 Gunnar Wagenknecht CLA 2020-12-21 08:54:35 EST
I managed to reproduce the issue on an U18 machine today. While reviewing the thread dump I noticed something:

main thread:
	...

	at org.eclipse.jface.text.contentassist.AdditionalInfoController$Timer.lambda$1(AdditionalInfoController.java:362)
	- locked <0x00000006ae746ec0> (a org.eclipse.jface.text.contentassist.AdditionalInfoController$1)


"Additional info timer" 
   java.lang.Thread.State: BLOCKED (on object monitor)
	at java.lang.Object.wait(java.base@11.0.9/Native Method)
	- waiting on <no object reference available>
	at org.eclipse.jface.text.contentassist.AdditionalInfoController$Timer.loop(AdditionalInfoController.java:314)
	- waiting to re-lock in wait() <0x00000006ae746ec0> (a org.eclipse.jface.text.contentassist.AdditionalInfoController$1)
	

Is this "waiting to re-lock in wait" problematic?

The issue seems to be specific to content assist.
Comment 36 Gunnar Wagenknecht CLA 2020-12-21 09:30:04 EST
(In reply to Gunnar Wagenknecht from comment #35)
> The issue seems to be specific to content assist.

It might be a red herring.


OS: Linux, v.4.15.0-128-generic, x86_64 / gtk 3.22.30, WebKit 2.30.3


The main thread is interesting, though:
- Display runAsyncMessages
- AdditionalInfoController (LEGACY_WAIT)
- makes BrowserInformationControl visible
- BrowserInformationControl spins SWT event loop to become visible
- this triggers OS.g_main_context_iteration
- OS.g_main_context_iteration fires key pressed event
- CompletionProposalPopup hide is requested
- its shell is disposed
- this triggers AbstractInformationControl dispose
- its shell is dsposed
- this triggers WebKit close
- WebKit close wants to execute JavaScript (why?)
- Webkit2AsyncToSync execAsyncAndWaitForReturn executes g_main_context_iteration
-> OS.g_main_context_iteration blocks (no timeout given)


The number of different events going on really indicates that this might be a concurrency issue. Not sure why it does not show up on other systems (Fedora). Perhaps the processing is different there.

The disposal of the browser happens while the browser information control is being made visible. Is this something to worry about?


What's also interesting is that it works fine for an hour and then deadlocks at the same place, which worked fine before.
Comment 37 Andrey Loskutov CLA 2020-12-21 11:03:41 EST
(In reply to Gunnar Wagenknecht from comment #36)
> 
> The number of different events going on really indicates that this might be
> a concurrency issue. Not sure why it does not show up on other systems
> (Fedora). Perhaps the processing is different there.

Can be webkit/GTK version specific. Both libraries are known to show various behavior changes (other would call it regressions) even at service releases.

> The disposal of the browser happens while the browser information control is
> being made visible. Is this something to worry about?

I believe GTK doesn't always like if we call it back in some SWT listeners, so it is possible. Typical "solution" is to decoupage the call from SWT to GTK via asyncExec from current GTK callback.
Comment 38 Gunnar Wagenknecht CLA 2021-02-19 12:16:23 EST
I have some good news to report.

A user built a webkit2gtk version from source without Debian/Ubuntu specific patches applied. This version makes Eclipse running without any issues on Ubuntu.

I believe I identified the problematic patch and built a new version without that patch applied.

Details here:
https://gist.github.com/guw/30bc8b14756e364c3d158fababbbd62d
Comment 39 Andrey Loskutov CLA 2021-02-19 12:28:58 EST
(In reply to Gunnar Wagenknecht from comment #38)
> I have some good news to report.
> 
> A user built a webkit2gtk version from source without Debian/Ubuntu specific
> patches applied. This version makes Eclipse running without any issues on
> Ubuntu.

Cool

> I believe I identified the problematic patch and built a new version without
> that patch applied.

I see "remove the line prefer-pthread.patch" sentence but I have no clue what this patch is about and if that is Ubuntu/Debian specific. Could you provide some pointers / links about this?

> Details here:
> https://gist.github.com/guw/30bc8b14756e364c3d158fababbbd62d

So far it reads as if Debian/Ubuntu webkit is broken. Is that true, or a similar patch need to be removed in other distributions (I'm thinking about RHEL/CentOS)?
Comment 40 Gunnar Wagenknecht CLA 2021-02-19 12:34:34 EST
(In reply to Andrey Loskutov from comment #39)
> I see "remove the line prefer-pthread.patch" sentence but I have no clue
> what this patch is about and if that is Ubuntu/Debian specific. Could you
> provide some pointers / links about this?

https://bugs.launchpad.net/ubuntu/+source/webkit2gtk/+bug/1916272

> > Details here:
> > https://gist.github.com/guw/30bc8b14756e364c3d158fababbbd62d
> 
> So far it reads as if Debian/Ubuntu webkit is broken. Is that true, or a
> similar patch need to be removed in other distributions (I'm thinking about
> RHEL/CentOS)?

Only the .deb packages are affected.
Comment 41 Gunnar Wagenknecht CLA 2021-03-06 07:26:56 EST
(In reply to Gunnar Wagenknecht from comment #38)
> I have some good news to report.

Declared victory too early.


> A user built a webkit2gtk version from source without Debian/Ubuntu specific
> patches applied. This version makes Eclipse running without any issues on
> Ubuntu.

The issue still occurs (just less frequent). What's also interesting - we have seen the frequency is high with 11.0.8 and much better with 11.0.9. But still occurs.

> I believe I identified the problematic patch and built a new version without
> that patch applied.

Red herring. :(
Comment 42 Gunnar Wagenknecht CLA 2021-03-06 08:23:48 EST
One thing that really bugs my ... why is Content Assist / BrowserInformationControl relying on JavaScript? Perhaps that is leading to the possibility here in the first place?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=571746
Comment 43 Stephan Herrmann CLA 2021-06-08 12:28:50 EDT
(In reply to Gunnar Wagenknecht from comment #36)
> The main thread is interesting, though:
> - Display runAsyncMessages
> - AdditionalInfoController (LEGACY_WAIT)
> - makes BrowserInformationControl visible
> - BrowserInformationControl spins SWT event loop to become visible
> - this triggers OS.g_main_context_iteration
> - OS.g_main_context_iteration fires key pressed event
> - CompletionProposalPopup hide is requested
> - its shell is disposed
> - this triggers AbstractInformationControl dispose
> - its shell is dsposed
> - this triggers WebKit close
> - WebKit close wants to execute JavaScript (why?)
> - Webkit2AsyncToSync execAsyncAndWaitForReturn executes
> g_main_context_iteration
> -> OS.g_main_context_iteration blocks (no timeout given)

This looks VERY similar to what is bugging me a LOT in bug 569668.

In that bug the symptom is not freeze but key events being processed out of order, due to Webkit2AsyncToSync.execAsyncAndWaitForReturn spinning the event loop while processing content assist. Result: content assist and direct key events hit the document in random order. This is happening many times per work day for me.

The other bug has a JUnit that reliably reproduces my problem on all my machines.

One reason the (my) issue is tricky to reproduce: the delayed showing of additional info has a finger in the pie, so you have to wait until this occurs, but then be quick.

However, so far I failed to reproduce when switching the GTK theme to Nextwaita-3.0. How can a GTK theme change the order of events in Eclipse? The best I can think of: some delay during painting via a given theme could skew the timing to make bugs appear more likely or less likely.

Anyway, it looks unhealthy to open a simple dispose() action in ways that creates all these bad side effects.

> - WebKit close wants to execute JavaScript (why?)

This made me scratch my head, too. Well this is the script in my case:
 "return SWTExecuteTemporaryFunctionCLOSE(window);"

Is this saying the only way to close a browser window is through javascript? And the only way to execute javascript is by spinning the main event loop?

Would it make sense to at least defer all this voodoo until regular events in the queue have been processed regularly?