Bug 205678 - [Clipboard] Using x11vnc with clipboard transfer causes eclipse to OOM
Summary: [Clipboard] Using x11vnc with clipboard transfer causes eclipse to OOM
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.4   Edit
Hardware: PC Linux-GTK
: P3 normal (vote)
Target Milestone: 4.5 M6   Edit
Assignee: Andrey Loskutov CLA
QA Contact:
URL:
Whiteboard:
Keywords: greatfix
: 206552 252692 366830 380631 449902 478144 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-10-07 23:01 EDT by Damien Mascord CLA
Modified: 2015-11-22 15:57 EST (History)
19 users (show)

See Also:


Attachments
Zipped log file during OOM (2.24 KB, application/x-zip-compressed)
2007-10-08 22:24 EDT, Damien Mascord CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Damien Mascord CLA 2007-10-07 23:01:09 EDT
Build ID: 3.3.0 I20070625 1500

Steps To Reproduce:
1. Open an x11vnc session with an already logged in X session (x11vnc -display :0)
2. Load eclipse
3. Use eclipse for a while, and right click on a few projects.
4. eclipse will quickly complain that it is out of memory and crash


More information:
1. This has happened with any 3.x.x release I have tried
2. If I disable clipboard transfer, it never crashes in this way.
3. I am running the vnc client on Windows XP Pro SP2.
4. The vnc server is Ubuntu Feisty Linux.
Comment 1 Remy Suen CLA 2007-10-08 19:43:24 EDT
Could you paste/attach a stacktrace or your log file?
Comment 2 Damien Mascord CLA 2007-10-08 22:24:32 EDT
Created attachment 79922 [details]
Zipped log file during OOM
Comment 3 Steve Northover CLA 2007-10-15 12:10:12 EDT
We need to either recreate the problem or insert debugging prints into the clipboard transfer to see how much memory is being requested.  Duong will follow up.
Comment 4 Steve Northover CLA 2007-10-15 12:11:46 EDT
The failure is in at org.eclipse.ui.part.ResourceTransfer.nativeToJava().  Are they asking for a huge buffer?
Comment 5 Duong Nguyen CLA 2007-10-16 19:14:13 EDT
The problem doesn't appear to be in an SWT class. It appears to be an eclipse UI problem.

Damien: Can you modify the ResourceTransfer class by applying the following patch? It will print the number of the resources that you're attempting to copy.

The ResourceTransfer class can be found by checking out the org.eclipse.ui.ide project from CVS. Let me know if you need help in doing this.

******** patch start *************
### Eclipse Workspace Patch 1.0
#P org.eclipse.ui.ide
Index: extensions/org/eclipse/ui/part/ResourceTransfer.java
===================================================================
RCS file: /cvsroot/eclipse/org.eclipse.ui.ide/extensions/org/eclipse/ui/part/ResourceTransfer.java,v
retrieving revision 1.5
diff -u -r1.5 ResourceTransfer.java
--- extensions/org/eclipse/ui/part/ResourceTransfer.java	8 May 2006 20:54:14 -0000	1.5
+++ extensions/org/eclipse/ui/part/ResourceTransfer.java	16 Oct 2007 23:07:16 -0000
@@ -158,6 +158,7 @@
                 new ByteArrayInputStream(bytes));
         try {
             int count = in.readInt();
+            System.out.println("ResourceTransfer: count=" + count); //$NON-NLS-1$
             IResource[] results = new IResource[count];
             for (int i = 0; i < count; i++) {
                 results[i] = readResource(in);
*********** patch end ****************
Comment 6 Steve Northover CLA 2007-10-18 14:22:02 EDT
*** Bug 206552 has been marked as a duplicate of this bug. ***
Comment 7 Damien Mascord CLA 2007-11-01 23:31:08 EDT
Heya,

Unfortunately, I haven't had the chance to do a recompile with this patch, would it be possible to put a jar attached to this Bug ?

Cheers,

Damien
Comment 8 Overmind DL1 CLA 2012-05-13 16:22:28 EDT
This bug is still occurring as of Eclipse 3.7.2-1.
Comment 9 Nicholas Lee CLA 2012-05-18 06:36:15 EDT
I also have this problem on startup of Eclipse CDT version 3.7.2, which can only be worked-around by disabling clipboard transfers in the VNC client (RealVNC). This is a real nuisance.

java.lang.OutOfMemoryError: Java heap space
	at org.eclipse.ui.part.ResourceTransfer.nativeToJava(ResourceTransfer.java:162)
	at org.eclipse.swt.dnd.Clipboard.getContents(Clipboard.java:323)
	at org.eclipse.swt.dnd.Clipboard.getContents(Clipboard.java:241)
	at org.eclipse.ui.views.navigator.PasteAction$1.run(PasteAction.java:194)
	at org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:180)
	at org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:150)
	at org.eclipse.swt.widgets.Display.syncExec(Display.java:4330)
	at org.eclipse.ui.views.navigator.PasteAction.updateSelection(PasteAction.java:189)
	at org.eclipse.ui.actions.BaseSelectionListenerAction.selectionChanged(BaseSelectionListenerAction.java:124)
	at org.eclipse.ui.views.navigator.RefactorActionGroup.updateActionBars(RefactorActionGroup.java:155)
	at org.eclipse.ui.views.navigator.MainActionGroup.updateActionBars(MainActionGroup.java:319)
	at org.eclipse.ui.views.navigator.ResourceNavigator.updateActionBars(ResourceNavigator.java:1412)
	at org.eclipse.ui.views.navigator.ResourceNavigator.createPartControl(ResourceNavigator.java:327)
	at org.eclipse.ui.internal.ViewReference.createPartHelper(ViewReference.java:375)
	at org.eclipse.ui.internal.ViewReference.createPart(ViewReference.java:229)
	at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
	at org.eclipse.ui.internal.PartPane.requestActivation(PartPane.java:263)
	at org.eclipse.ui.internal.presentations.PresentablePart.setFocus(PresentablePart.java:192)
	at org.eclipse.ui.internal.presentations.util.TabbedStackPresentation$1.handleEvent(TabbedStackPresentation.java:94)
	at org.eclipse.ui.internal.presentations.util.AbstractTabFolder.fireEvent(AbstractTabFolder.java:269)
	at org.eclipse.ui.internal.presentations.util.AbstractTabFolder.fireEvent(AbstractTabFolder.java:274)
	at org.eclipse.ui.internal.presentations.util.AbstractTabFolder.handleMouseDown(AbstractTabFolder.java:344)
	at org.eclipse.ui.internal.presentations.util.AbstractTabFolder$3.mouseDown(AbstractTabFolder.java:78)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:191)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1258)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3588)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3209)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2701)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2665)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2499)
	at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:679)
Comment 10 Anton Leherbauer CLA 2012-05-30 03:27:20 EDT
*** Bug 380631 has been marked as a duplicate of this bug. ***
Comment 11 Andrey Loskutov CLA 2012-10-15 08:19:13 EDT
We can reproduce it on RHEL 5.8 and Eclipse 3.7.2 64 bit

* start with a clean system: freshly rebooted linux system
* start x11vnc
* start Eclipse
* perform any copy/paste operation (some text) inside your vnc client
* operate the context menu in the project explorer in Eclipse
* Eclipse get OOM

Either it is a programming defect on Eclipse side (wrong pre-condition / wrong assumption about the clipboard content) or an issue of the clipboard data objects delivered by x11vnc. In detail, Eclipse assumes that the clipboard content describes a bunch of selected "resources" (files) and misinterprets the binary format. This leads to a high random integer number of Resource objects being created and leads to the out of memory situation (the first four bytes of the clipboard content are taken as integer value. This represents a positive or negative number up to 2^31).

To workaround the problem, one can start x11vnc with command line options: -noclipboard and -nosetclipboard

To fix the issue, org.eclipse.ui.views.navigator.PasteAction and all of it's  derivatives (org.eclipse.cdt.internal.ui.cview.PasteAction, org.eclipse.ui.internal.navigator.resources.actions.PasteAction) should be fixed.

The fix could be (not tested):

change this:

        shell.getDisplay().syncExec(new Runnable() {
            public void run() {
                // clipboard must have resources or files
                ResourceTransfer resTransfer = ResourceTransfer.getInstance();
                clipboardData[0] = (IResource[]) clipboard
                        .getContents(resTransfer);
            }
        });

to this:

        shell.getDisplay().syncExec(new Runnable() {
            public void run() {
                // clipboard must have resources or files
                ResourceTransfer resTransfer = ResourceTransfer.getInstance();
                // bute better to test it first
                if (isAvailable(resTransfer, clipboard.getAvailableTypes())) {
                    clipboardData[0] = (IResource[]) clipboard
                        .getContents(resTransfer);
                }
            }
        });


	private static boolean isAvailable(Transfer transfer,
			TransferData[] availableDataTypes) {
		for (int i = 0; i < availableDataTypes.length; i++) {
			if (transfer.isSupportedType(availableDataTypes[i]))
				return true;
		}
		return false;
	}
Comment 12 Eric Moffatt CLA 2012-10-15 14:40:26 EDT
*** Bug 252692 has been marked as a duplicate of this bug. ***
Comment 13 Martin Oberhuber CLA 2013-11-11 07:17:21 EST
CQ:WIND00443016

We still see the problem with Eclipse 3.8.2 / CDT 8.2.0
on some Ubuntu 12.04 machines.
Comment 14 Neven Boric CLA 2014-08-28 15:19:52 EDT
I had to resort to this:

#!/bin/sh
while true; do
	xclip -selection c -d :0 -o | xclip -selection c -d :0
	sleep 1
done

Problem "solved"
Comment 15 Martin Oberhuber CLA 2014-08-29 01:42:37 EDT
Interesting - thanks very much for this suggestion !

So with some background program, you read the clipboard selection and set it again every second. 

If that really improves things for Eclipse, it implies that xclip provides a "better" clipboard than whatever external application used to provide the clipboard and was not responsive ; and, it also implies that xclip can read the clipboard in a "better" way than Eclipse.

For how long have you actually been using this workaround, and what are your experiences ? I'd assume that it resolves the problem to a maximum 1 second wait time ... have you seen any other side effects such as sometimes not being able to copy-and-paste ?
Comment 16 Andrey Loskutov CLA 2014-08-29 01:47:57 EDT
Folks, what is about solution from comment 11?
All what we need is someone from platform UI team who can test and apply the patch. There is no black magic needed.
Comment 17 Neven Boric CLA 2014-08-29 13:02:28 EDT
I have been using it for about a week now, Eclipse hasn't crashed once and I haven't seen any problem with the clipboard in any direction. I'm fully aware how horrible hack it is, just a workaround until a proper solution.

(In reply to Martin Oberhuber from comment #15)
> For how long have you actually been using this workaround, and what are your
> experiences ? I'd assume that it resolves the problem to a maximum 1 second
> wait time ... have you seen any other side effects such as sometimes not
> being able to copy-and-paste ?
Comment 18 Martin Oberhuber CLA 2014-09-05 05:28:19 EDT
(In reply to Andrey Loskutov from comment #16)
CQ:WIND00-WB4-51

I agree with Andrey. The issue is real, and the solution from comment 11 makes sense and should be applied. Could any committer step forward and merge the patch ? Or would submissions on Gerrit be required for IP signoff ?
Comment 19 Benjamin Shadwick CLA 2014-10-17 00:50:08 EDT
I'm experiencing the issue in a VNC session to an Xubuntu 14.01.1 LTS box with Luna running via either openjdk or Oracle Java 8.

I'm not using autocutsel, but I did notice that the parameters that x11vnc is launched with seem to be a factor.

This results in the error:
/usr/bin/x11vnc -xkb -auth /var/run/lightdm/root/:0 -nowireframe -wirecopyrect top -wmdt xfce -threads -repeat -rfbauth /etc/x11vnc.pass -shared -forever -rfbport 5900 -o /var/log/x11vnc.log

This does not:
/usr/bin/x11vnc -xkb -auth /var/run/lightdm/root/:0 -noxrecord -noxfixes -noxdamage -rfbauth /etc/x11vnc.pass -forever -bg -rfbport 5900 -o /var/log/x11vnc.log
Comment 20 Benjamin Shadwick CLA 2014-10-17 01:27:16 EDT
Update:

Ignore my previous comment. I was able to reproduce it with both settings. It seems that restarting x11vnc sometimes prevents the error from happening until eclipse has been restarting a couple times with new things in the clipboard.
Comment 21 Andrey Loskutov CLA 2015-01-20 18:16:29 EST
OK, I have a patch and I've chosen different approach: limit the number of resources we can paste from clipboard. 

No one seriously would copy/paste or drag/drop more then 1.000.000
resources: only creating an *empty* array of 100.000.000 resources will
cause OOME on 512 MB heap size (default for shipped Eclipse packages),
same with copy/paste of a *full* array of 10.000.000 elements.

The patch introduces a maximum limit for ResourceTransfer of 1.000.000
elements. In case the elements count exceed this limit, an error message
will be logged to the error log and the transfer operation will be
cancelled. JUnit test for the new limit is added too.

Patch: https://git.eclipse.org/r/39996
Comment 23 Lars Vogel CLA 2015-02-16 07:01:52 EST
Thanks Andrey, change looks fine to me. WHoever tries to copy 1000 000 items should now get a reasonable error message.
Comment 24 David Williams CLA 2015-02-17 03:01:51 EST
FYI, see bug 460069 for some JavaDoc work that needs to be fixed.
Comment 25 Dani Megert CLA 2015-02-17 04:22:44 EST
This causes a test failure in our official build, which reports a Javadoc issue:
http://download.eclipse.org/eclipse/downloads/drops4/N20150216-2000/testresults/html/org.eclipse.releng.tests_linux.gtk.x86_64_8.0.html

==>

http://download.eclipse.org/eclipse/downloads/drops4/N20150216-2000/compilelogs/platform.doc.isv.javadoc.txt

../../../eclipse.platform.ui/bundles/org.eclipse.ui.ide/extensions/org/eclipse/ui/part/ResourceTransfer.java:59: warning - #MAX_RESOURCES_TO_TRANSFER (referenced by @value tag) is an unknown reference.
1 warning
Comment 26 Lars Vogel CLA 2015-02-17 04:44:30 EST
(In reply to Dani Megert from comment #25)
> This causes a test failure in our official build, which reports a Javadoc
> issue:
> http://download.eclipse.org/eclipse/downloads/drops4/N20150216-2000/
> testresults/html/org.eclipse.releng.tests_linux.gtk.x86_64_8.0.html

See https://bugs.eclipse.org/bugs/show_bug.cgi?id=205678#c24, we are solving this via Bug 460069.
Comment 27 Martin Oberhuber CLA 2015-06-10 00:23:43 EDT
(In reply to Lars Vogel from comment #23)
> Thanks Andrey, change looks fine to me. WHoever tries to copy 1000 000 items
> should now get a reasonable error message.

I think the message could be further improved, in order to hint users to the specific problem with x11vnc. For example, on an embedded Yocto Linux 64bit system, I had been running into OOME with Luna, then later with Mars I got this in the errorlog (hyperlinks are to detailed system info + stacks from AERI):

“Transfer aborted, too many resources: 1954115685”
https://dev.eclipse.org/recommenders/committers/confess/#/incidents/55717524e4b018246752319d
https://dev.eclipse.org/recommenders/committers/confess/#/incidents/55717524e4b018246752319e

The error is only logged, but due to the Automated Error Reporting Initiative (AERI) it is still very visible and made me feel my install was broken. It was pure luck that I figured out comment 11 or comment 14 here could be used as a workaround.

Any thoughts / ideas for a better error message ?
Comment 28 Dani Megert CLA 2015-06-10 03:07:40 EDT
(In reply to Martin Oberhuber from comment #27)
> (In reply to Lars Vogel from comment #23)
> > Thanks Andrey, change looks fine to me. WHoever tries to copy 1000 000 items
> > should now get a reasonable error message.
> 
> I think the message could be further improved, in order to hint users to the
> specific problem with x11vnc. For example, on an embedded Yocto Linux 64bit
> system, I had been running into OOME with Luna, then later with Mars I got
> this in the errorlog (hyperlinks are to detailed system info + stacks from
> AERI):
> 
> “Transfer aborted, too many resources: 1954115685”
> https://dev.eclipse.org/recommenders/committers/confess/#/incidents/55717524e4b018246752319d
> 
> https://dev.eclipse.org/recommenders/committers/confess/#/incidents/55717524e4b018246752319e
> 
> 
> The error is only logged, but due to the Automated Error Reporting
> Initiative (AERI) it is still very visible and made me feel my install was
> broken. It was pure luck that I figured out comment 11 or comment 14 here
> could be used as a workaround.
> 
> Any thoughts / ideas for a better error message ?

I suggest you open a new bug with a proposal.
Comment 29 Andrey Loskutov CLA 2015-06-23 11:47:08 EDT
*** Bug 449902 has been marked as a duplicate of this bug. ***
Comment 30 Andrey Loskutov CLA 2015-09-23 05:36:51 EDT
*** Bug 478144 has been marked as a duplicate of this bug. ***
Comment 31 Andrey Loskutov CLA 2015-10-02 07:49:36 EDT
*** Bug 366830 has been marked as a duplicate of this bug. ***
Comment 32 Eclipse Genie CLA 2015-11-17 08:40:46 EST
New Gerrit change created: https://git.eclipse.org/r/60609
Comment 33 Andrey Loskutov CLA 2015-11-17 09:23:27 EST
(In reply to Martin Oberhuber from comment #27)
> I think the message could be further improved, in order to hint users to the
> specific problem with x11vnc. 
> [...]
> Any thoughts / ideas for a better error message ?

https://git.eclipse.org/r/60609 ?
Comment 34 Brian de Alwis CLA 2015-11-18 11:38:02 EST
I'm late to this party, but is the solution in comment 11 not workable?

Looking at the OS X implementation of Clipboard#getContents, it looks like it has some protection to only pull out data if the transfer is compatible.
Comment 35 Andrey Loskutov CLA 2015-11-18 11:46:26 EST
(In reply to Brian de Alwis from comment #34)
> I'm late to this party, but is the solution in comment 11 not workable?
> 
> Looking at the OS X implementation of Clipboard#getContents, it looks like
> it has some protection to only pull out data if the transfer is compatible.

This didn't worked out since the corrupted clipboard contain expected content type.