Bug 196664 - [refresh][performance] Refresh queries too many parents
Summary: [refresh][performance] Refresh queries too many parents
Status: CLOSED WORKSFORME
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: 2.0   Edit
Hardware: PC Windows XP
: P2 normal (vote)
Target Milestone: 2.0.1   Edit
Assignee: David McKnight CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks: 191374
  Show dependency tree
 
Reported: 2007-07-16 11:05 EDT by Martin Oberhuber CLA
Modified: 2008-01-23 13:20 EST (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Oberhuber CLA 2007-07-16 11:05:02 EDT
RSE CVS HEAD of 16-Jul-2007.
With an SSH connection and Tracing enabled, expand 
    My Home/RSETest/1.0.1/eclipse/plugins
with the "plugins" folder selected, right-click > Refresh

--> Tracing shows:
2007-07-16 16:35:59.219 | Worker-6 | SftpFileService.internalFetch: /folk/mober/RSETest/1.0.1/eclipse/plugins
2007-07-16 16:36:05.672 | main | SftpFileService.getFile: plugins

with the "plugins" folder still selected, right-click > Refresh again
--> Tracing shows:
2007-07-16 16:36:05.719 | Worker-6 | SftpFileService.internalFetch: /folk/mober/RSETest/1.0.1/eclipse/plugins
2007-07-16 16:36:38.047 | main | SftpFileService.getFile: plugins
2007-07-16 16:36:38.047 | main | SftpFileService.getFile: eclipse

with the "plugins" folder still selected, right-click > Refresh again
--> Tracing shows:
2007-07-16 16:36:38.078 | Worker-6 | SftpFileService.internalFetch: /folk/mober/RSETest/1.0.1/eclipse/plugins
2007-07-16 16:37:00.922 | main | SftpFileService.getFile: plugins
2007-07-16 16:37:00.937 | main | SftpFileService.getFile: eclipse
2007-07-16 16:37:00.937 | main | SftpFileService.getFile: 1.0.1
2007-07-16 16:37:00.937 | main | SftpFileService.getFile: RSETest
2007-07-16 16:37:00.937 | main | SftpFileService.getFile: mober
2007-07-16 16:37:00.937 | main | SftpFileService.getFile: folk

Apparently, every refresh query leads to "getFile" queries further up in the remote file system tree. This may be coming from a problem with markStale() on the parent folder and not marking it non-stale after the query again. Expected behaviro would be that "Refresh" on a given folder always performs exactly the same queries on the remote.

Good and reproducable performance on Refresh is very important, especially for EFS. See also bug #191374. We've had a similar issue and discussions regarding markStale() already on bug #188160 and bug #173518 comment 38 onwards. 

What's more, refresh must not happen on the dispatch thread ("main"), this is covered by bug #196662. The backtrace on that bug seems to indicate that the problem is in SystemView$ResourceChangedJob:

SftpFileService.getFile(String, String, IProgressMonitor) line: 250     
FileServiceSubSystem.getFile(String, String, IProgressMonitor) line: 298        
FileServiceSubSystem.getRemoteFileObject(String, IProgressMonitor) line: 220    
FileServiceSubSystem(RemoteFileSubSystem).getObjectWithAbsoluteName(String)
line: 1059  
SystemView$ResourceChangedJob.runInUIThread(IProgressMonitor) line: 2164        

-----------Enter bugs above this line-----------
TM 2.0
installation : eclipse-SDK-3.3 (I20070625-1500), cdt-4.0.0, emf-2.3.0
RSE install  : workspace HEAD
java.runtime : Sun 1.6.0_01-b06
os.name:     : Windows XP 5.1, Service Pack 1
------------------------------------------------
systemtype   : Windows-local, Dstore-win, Dstore-linux
targetos     : Red Hat Enterprise Linux WS release 4 (Nahant Update 3)
targetuname  : Linux parser 2.6.9-34.EL #1 i686 athlon i386 GNU/Linux
targetvm     : Sun Java HotSpot(TM) Client VM (build 1.4.2_12-b03, mixed mode)
------------------------------------------------
Comment 1 Martin Oberhuber CLA 2007-07-16 11:14:00 EDT
For another discussion on markStale(), see also bug 189265 comment 14
Comment 2 David McKnight CLA 2007-07-19 11:48:20 EDT
One issue that comes up with refresh is that a query can happen when we cache IRemoteFiles:

Thread [Worker-0] (Suspended (breakpoint at line 173 in RemoteFile))	
	SftpRemoteFile(RemoteFile).getParentRemoteFile() line: 173	
	FileServiceSubSystem(RemoteFileSubSystem).cacheRemoteFile(IRemoteFile, String) line: 1262	
	FileServiceSubSystem(RemoteFileSubSystem).cacheRemoteFile(IRemoteFile) line: 1283	
	SftpFileAdapter.convertToRemoteFiles(FileServiceSubSystem, IRemoteFileContext, IRemoteFile, IHostFile[]) line: 40	
	FileServiceSubSystem.listFoldersAndFiles(IRemoteFile, String, IRemoteFileContext, IProgressMonitor) line: 333	
	FileServiceSubSystem(RemoteFileSubSystem).listFoldersAndFiles(IRemoteFile, String, IProgressMonitor) line: 948	
	FileServiceSubSystem(RemoteFileSubSystem).internalResolveFilterString(String, IProgressMonitor) line: 672	
	FileServiceSubSystem(RemoteFileSubSystem).internalResolveFilterStrings(String[], IProgressMonitor) line: 450	
	FileServiceSubSystem(SubSystem).resolveFilterStrings(String[], IProgressMonitor) line: 2067	
	SystemViewFilterReferenceAdapter.internalGetChildren(Object, IProgressMonitor) line: 416	
	SystemViewFilterReferenceAdapter.getChildren(IAdaptable, IProgressMonitor) line: 257	
	SystemViewFilterReferenceAdapter.getChildren(IContextObject, IProgressMonitor) line: 265	
	SystemFetchOperation.execute(IProgressMonitor) line: 265	
	SystemFetchOperation.run(IProgressMonitor) line: 128	
	SystemViewFilterReferenceAdapter(AbstractSystemViewAdapter).fetchDeferredChildren(Object, IElementCollector, IProgressMonitor) line: 1970	
	DeferredTreeContentManager$1.run(IProgressMonitor) line: 196	
	Worker.run() line: 55	

In this case, the query is useless because we're just doing this for caching purposes and no parent remote file was there in the first place:
		
// replace file under parent
if (oldFile != null && oldFile.getParentRemoteFile() != null)
{
 oldFile.getParentRemoteFile().replaceContent(oldFile, file);
}

This however is different from the stack you're providing which is the result of the remote refresh event.
 
Comment 3 David McKnight CLA 2007-07-25 14:44:27 EDT
I've put in a fix to avoid the query of the parents when caching.   Martin, are you still seeing this problem?  I've tried in the debugger, but I'm not able to get those calls to getFile() that you found in your trace.  Do you still see this?
Comment 4 David McKnight CLA 2007-07-26 11:47:25 EDT
I'm not seeing this so I'm going to mark it worksforme.
Comment 5 Martin Oberhuber CLA 2008-01-23 13:20:33 EST
Verified with 3.0M4