Bug 160084

Summary: RSE is hanging eclipse and slowing down my system dramatically
Product: [Tools] Target Management Reporter: Michael Scharf <eclipse>
Component: RSEAssignee: David McKnight <dmcknigh>
Status: RESOLVED FIXED QA Contact: Martin Oberhuber <mober.at+eclipse>
Severity: critical    
Priority: P1 CC: ddykstal.eclipse
Version: unspecified   
Target Milestone: 1.0   
Hardware: PC   
OS: Windows XP   
Whiteboard:
Attachments:
Description Flags
The eclipse Thread Dump
none
another bad stack trace none

Description Michael Scharf CLA 2006-10-06 15:48:57 EDT
After running RSE for some hours, eclipse started using more and more CPU time. In the Progress bar a lot "Resolve Filter String...Operation" jobs appeared (I was not able to make a screenshot, because my system became unusably slow even though I set the priority of the java process to idle). 

However, I was able to create a thread dump. There is a very deep recursion in RSE/SWT in the main thread.


	- locked <0x1db0d4d0> (a org.eclipse.swt.widgets.RunnableLock)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3325)
	at org.eclipse.swt.widgets.Display.msgFilterProc(Display.java:2758)
	at org.eclipse.swt.internal.win32.OS.TrackPopupMenu(Native Method)
	at org.eclipse.swt.widgets.Menu._setVisible(Menu.java:225)
	at org.eclipse.swt.widgets.Display.runPopups(Display.java:3374)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2962)
	at org.eclipse.rse.core.subsystems.SubSystem.scheduleJob(SubSystem.java:2110)
	at org.eclipse.rse.core.subsystems.SubSystem.resolveFilterStrings(SubSystem.java:2058)
	at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.internalGetChildren(SystemViewFilterReferenceAdapter.java:363)
	- locked <0x17e36630> (a org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter)
	at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.getChildren(SystemViewFilterReferenceAdapter.java:240)
	at org.eclipse.rse.ui.view.SystemTableTreeViewProvider.getElements(SystemTableTreeViewProvider.java:165)
	at org.eclipse.jface.viewers.StructuredViewer.getRawChildren(StructuredViewer.java:927)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getRawChildren(AbstractTreeViewer.java:1144)
	at org.eclipse.jface.viewers.TreeViewer.getRawChildren(TreeViewer.java:768)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getFilteredChildren(AbstractTreeViewer.java:574)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getSortedChildren(AbstractTreeViewer.java:543)
	at org.eclipse.jface.viewers.AbstractTreeViewer.updateChildren(AbstractTreeViewer.java:2337)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefreshStruct(AbstractTreeViewer.java:1651)
	at org.eclipse.jface.viewers.TreeViewer.internalRefreshStruct(TreeViewer.java:955)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1627)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1582)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1568)
	at org.eclipse.rse.ui.view.SystemTableTreeView.systemResourceChanged(SystemTableTreeView.java:821)
	at org.eclipse.rse.internal.model.SystemResourceChangeManager.notify(SystemResourceChangeManager.java:74)
	at org.eclipse.rse.model.SystemRegistry.fireEvent(SystemRegistry.java:2803)
	at org.eclipse.rse.ui.view.monitor.MonitorViewPage$2.run(MonitorViewPage.java:189)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:123)
	- locked <0x1db0d560> (a org.eclipse.swt.widgets.RunnableLock)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3325)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2971)
	at org.eclipse.rse.core.subsystems.SubSystem.scheduleJob(SubSystem.java:2110)
	at org.eclipse.rse.core.subsystems.SubSystem.resolveFilterStrings(SubSystem.java:2058)
	at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.internalGetChildren(SystemViewFilterReferenceAdapter.java:363)
	- locked <0x17e36630> (a org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter)
	at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.getChildren(SystemViewFilterReferenceAdapter.java:240)
	at org.eclipse.rse.ui.view.SystemTableTreeViewProvider.getElements(SystemTableTreeViewProvider.java:165)
	at org.eclipse.jface.viewers.StructuredViewer.getRawChildren(StructuredViewer.java:927)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getRawChildren(AbstractTreeViewer.java:1144)
	at org.eclipse.jface.viewers.TreeViewer.getRawChildren(TreeViewer.java:768)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getFilteredChildren(AbstractTreeViewer.java:574)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getSortedChildren(AbstractTreeViewer.java:543)
	at org.eclipse.jface.viewers.AbstractTreeViewer.updateChildren(AbstractTreeViewer.java:2337)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefreshStruct(AbstractTreeViewer.java:1651)
	at org.eclipse.jface.viewers.TreeViewer.internalRefreshStruct(TreeViewer.java:955)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1627)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1582)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1568)
	at org.eclipse.rse.ui.view.SystemTableTreeView.systemResourceChanged(SystemTableTreeView.java:821)
	at org.eclipse.rse.internal.model.SystemResourceChangeManager.notify(SystemResourceChangeManager.java:74)
	at org.eclipse.rse.model.SystemRegistry.fireEvent(SystemRegistry.java:2803)
	at org.eclipse.rse.ui.view.monitor.MonitorViewPage$2.run(MonitorViewPage.java:189)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:123)
	- locked <0x1db04788> (a org.eclipse.swt.widgets.RunnableLock)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3325)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2971)
	at org.eclipse.rse.core.subsystems.SubSystem.scheduleJob(SubSystem.java:2110)
	at org.eclipse.rse.core.subsystems.SubSystem.resolveFilterStrings(SubSystem.java:2058)
	at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.internalGetChildren(SystemViewFilterReferenceAdapter.java:363)
	- locked <0x17e36630> (a org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter)
	at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.getChildren(SystemViewFilterReferenceAdapter.java:240)
	at org.eclipse.rse.ui.view.SystemTableTreeViewProvider.getElements(SystemTableTreeViewProvider.java:165)
	at org.eclipse.jface.viewers.StructuredViewer.getRawChildren(StructuredViewer.java:927)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getRawChildren(AbstractTreeViewer.java:1144)
	at org.eclipse.jface.viewers.TreeViewer.getRawChildren(TreeViewer.java:768)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getFilteredChildren(AbstractTreeViewer.java:574)
	at org.eclipse.jface.viewers.AbstractTreeViewer.getSortedChildren(AbstractTreeViewer.java:543)
	at org.eclipse.jface.viewers.AbstractTreeViewer.updateChildren(AbstractTreeViewer.java:2337)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefreshStruct(AbstractTreeViewer.java:1651)
	at org.eclipse.jface.viewers.TreeViewer.internalRefreshStruct(TreeViewer.java:955)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1627)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1582)
	at org.eclipse.jface.viewers.AbstractTreeViewer.internalRefresh(AbstractTreeViewer.java:1568)
	at org.eclipse.rse.ui.view.SystemTableTreeView.systemResourceChanged(SystemTableTreeView.java:821)
	at org.eclipse.rse.internal.model.SystemResourceChangeManager.notify(SystemResourceChangeManager.java:74)
	at org.eclipse.rse.model.SystemRegistry.fireEvent(SystemRegistry.java:2803)
	at org.eclipse.rse.ui.view.monitor.MonitorViewPage$2.run(MonitorViewPage.java:189)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:123)
	- locked <0x1da21798> (a org.eclipse.swt.widgets.RunnableLock)
Comment 1 Michael Scharf CLA 2006-10-06 15:50:02 EDT
Created attachment 51575 [details]
The eclipse Thread Dump

Here is the thread dump of eclipse....
Comment 2 Michael Scharf CLA 2006-10-06 21:52:47 EDT
See bug 160111 for some more details.
Comment 3 Martin Oberhuber CLA 2006-10-07 16:21:04 EDT
Dave- I consider this critical, and bug 160111 has some details that should help finding the problem quickly.
Comment 4 David McKnight CLA 2006-10-10 11:34:43 EDT
This looks like it may be a duplicate of this one: https://bugs.eclipse.org/bugs/show_bug.cgi?id=150954

I'm assuming that the monitor view is polling when this happens.  Is this correct?
Comment 5 Michael Scharf CLA 2006-10-10 12:01:52 EDT
I don't remember. But I see *lots* of problems related to the nested event loop. At the moment my eclipse does not exit because of this stack trace.

The more I look into the code the more I get scared about threading and nested event loop issues. Does anybody really understand what is happening with treading? 

Just open a few connections and start clicking fast on some directories (to open), run a few monitors, run remote searches and you are guaranteed to hang. Actually I am having a hard time to find a "path" for my demo on Thursday where I don't hang RSE.

	at java.lang.Object.wait(Native Method)
	at org.eclipse.rse.services.dstore.util.DStoreStatusMonitor.waitForUpdate(DStoreStatusMonitor.java:383)
	- locked <0x151775b8> (a org.eclipse.rse.services.dstore.util.DStoreStatusMonitor)
	at org.eclipse.rse.services.dstore.util.DStoreStatusMonitor.waitForUpdate(DStoreStatusMonitor.java:308)
	- locked <0x151775b8> (a org.eclipse.rse.services.dstore.util.DStoreStatusMonitor)
	at org.eclipse.rse.services.dstore.util.DStoreStatusMonitor.waitForUpdate(DStoreStatusMonitor.java:252)
	at org.eclipse.rse.services.dstore.AbstractDStoreService.dsQueryCommand(AbstractDStoreService.java:87)
	at org.eclipse.rse.services.dstore.files.DStoreFileService.fetch(DStoreFileService.java:1210)
	at org.eclipse.rse.services.dstore.files.DStoreFileService.getFiles(DStoreFileService.java:1162)
	at org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem.getFiles(FileServiceSubSystem.java:297)
	at org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem.listFiles(FileServiceSubSystem.java:373)
	at org.eclipse.rse.subsystems.files.core.subsystems.RemoteFileSubSystem.listFiles(RemoteFileSubSystem.java:896)
	at org.eclipse.rse.subsystems.files.core.subsystems.RemoteFileSubSystem.listFiles(RemoteFileSubSystem.java:880)
	at org.eclipse.rse.subsystems.files.dstore.subsystem.DStoreFileSubSystemSearchResultConfiguration.getResults(DStoreFileSubSystemSearchResultConfiguration.java:83)
	at org.eclipse.rse.services.search.HostSearchResultSet.getAllResults(HostSearchResultSet.java:117)
	at org.eclipse.rse.ui.view.search.SystemSearchTableView.systemResourceChanged(SystemSearchTableView.java:302)
	at org.eclipse.rse.internal.model.SystemResourceChangeManager.notify(SystemResourceChangeManager.java:74)
	at org.eclipse.rse.model.SystemRegistry.fireEvent(SystemRegistry.java:2803)
	at org.eclipse.rse.subsystems.files.core.servicesubsystem.OutputRefresh.run(OutputRefresh.java:43)
	at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
	at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:123)
	- locked <0x107c0a40> (a org.eclipse.swt.widgets.RunnableLock)
	at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3325)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:2971)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:1914)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:1878)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:419)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.IDEApplication.run(IDEApplication.java:95)
	at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.core.launcher.Main.invokeFramework(Main.java:336)
	at org.eclipse.core.launcher.Main.basicRun(Main.java:280)
	at org.eclipse.core.launcher.Main.run(Main.java:977)
	at org.eclipse.core.launcher.Main.main(Main.java:952)
Comment 6 Michael Scharf CLA 2006-10-10 12:14:49 EDT
Created attachment 51710 [details]
another bad stack trace

My system does not hang, but eclipse is blocking and hanging.
Comment 7 Michael Scharf CLA 2006-10-10 12:26:46 EDT
plus it cannot exit: I have to kill eclipse to exit.

BTW I use the following too to get stack traces of a running eclipse
  http://www.adaptj.com/root/main/stacktrace
Webstart:
  http://tmitevski.users.mcs2.netarray.com/stacktrace/app/launch.jnlp
  
Comment 8 David McKnight CLA 2006-10-10 12:29:45 EDT
I've got a fix for this now.  We were firing a EVENT_REFRESH after doing a query which causes views to reissue the query.  I've changed the event to EVENT_CHANGE_CHILDREN which just deals with redrawing.
Comment 9 Michael Scharf CLA 2006-10-10 12:33:09 EDT
I'll test it as soon as the build is available...
The general problem of the nested event loops will be addressed in bug 160111?
Comment 10 David McKnight CLA 2006-10-10 12:40:44 EDT
I just saw your other comments here which would be separate issues I think from
the monitor one.  Could you open a separate defect for each of those separate scenarios (ones that don't involve the monitor view polling)?
Comment 11 Martin Oberhuber CLA 2008-08-13 13:07:52 EDT
[target cleanup] 1.0 RC2 was the original target milestone for this bug