Community
Participate
Working Groups
Reproduced with an ssh connection on Windows, but supposedly happens with other connection types too: Create a new filter for a remote directory e.g. on build.eclipse.org, "/home/data/users/moberhuber/downloads-tm/downloads/drops/N20060627-1258" Select the filter and choose Refresh --> See the contents Select the filter and choose "Show in Table" --> See the contents Select the filter and choose Refresh --> now, two refresh jobs are scheduled as the following backtrace shows. This can be seen by placing a breakpoint in SftpFileService.getFile() or SftpFileService.internalFetch(). Thread [Worker-36] (Suspended (breakpoint at line 140 in SftpFileService)) SftpFileService.getFile(IProgressMonitor, String, String) line: 140 FileServiceSubSystem.getFile(IProgressMonitor, String, String) line: 287 FileServiceSubSystem.getRemoteFileObject(String) line: 209 FileServiceSubSystem(RemoteFileSubSystem).internalResolveFilterString(IProgressMonitor, String) line: 588 FileServiceSubSystem(RemoteFileSubSystem).internalResolveFilterStrings(IProgressMonitor, String[]) line: 414 SubSystem$ResolveAbsolutesJob.performOperation(IProgressMonitor) line: 1424 SubSystem$ResolveAbsolutesJob(SubSystem$SubSystemOperationJob).run(IProgressMonitor) line: 1332 Worker.run() line: 58 Thread [Worker-39] (Running) Thread [Worker-41] (Suspended (breakpoint at line 140 in SftpFileService)) SftpFileService.getFile(IProgressMonitor, String, String) line: 140 FileServiceSubSystem.getFile(IProgressMonitor, String, String) line: 287 FileServiceSubSystem.getRemoteFileObject(String) line: 209 FileServiceSubSystem(RemoteFileSubSystem).internalResolveFilterString(IProgressMonitor, String) line: 588 FileServiceSubSystem(RemoteFileSubSystem).internalResolveFilterStrings(IProgressMonitor, String[]) line: 414 FileServiceSubSystem(SubSystem).resolveFilterStrings(IProgressMonitor, String[]) line: 2214 SystemViewFilterReferenceAdapter.internalGetChildren(IProgressMonitor, Object) line: 367 SystemViewFilterReferenceAdapter.getChildren(IProgressMonitor, Object) line: 226 SystemFetchOperation.execute(IProgressMonitor) line: 175 SystemFetchOperation.run(IProgressMonitor) line: 92 SystemViewFilterReferenceAdapter(AbstractSystemViewAdapter).fetchDeferredChildren(Object, IElementCollector, IProgressMonitor) line: 1766 DeferredTreeContentManager$1.run(IProgressMonitor) line: 207 Worker.run() line: 58
Note that the multiple refresh jobs are problematic if the file service is single threaded, thus they should be avoided.
I was able to reproduce this problem with SftpFileService but not the others. With Sftp, I'm always getting this stack: at com.jcraft.jsch.ChannelSftp.stat(Unknown Source) at com.jcraft.jsch.ChannelSftp.ls(Unknown Source) at org.eclipse.rse.services.ssh.files.SftpFileService.internalFetch(SftpFileService.java:175) at org.eclipse.rse.services.files.AbstractFileService.getFilesAndFolders(AbstractFileService.java:45) at org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem.getFilesAndFolders(FileServiceSubSystem.java:282) at org.eclipse.rse.subsystems.files.core.servicesubsystem.FileServiceSubSystem.listFoldersAndFiles(FileServiceSubSystem.java:316) at org.eclipse.rse.subsystems.files.core.subsystems.RemoteFileSubSystem.listFoldersAndFiles(RemoteFileSubSystem.java:907) at org.eclipse.rse.subsystems.files.core.subsystems.RemoteFileSubSystem.internalResolveFilterString(RemoteFileSubSystem.java:637) at org.eclipse.rse.subsystems.files.core.subsystems.RemoteFileSubSystem.internalResolveFilterStrings(RemoteFileSubSystem.java:414) at org.eclipse.rse.core.subsystems.SubSystem.resolveFilterStrings(SubSystem.java:2214) at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.internalGetChildren(SystemViewFilterReferenceAdapter.java:367) at org.eclipse.rse.ui.view.SystemViewFilterReferenceAdapter.getChildren(SystemViewFilterReferenceAdapter.java:226)
The problem is reproducable with dstore as well: * place a breakpoint in DStoreFileService.getFile() * In the Preferences, ensure that "Use deferred queries" Preference is ON * Create a new filter e.g. "/folk/mober/mydir" * Expand the filter * Select the filter and choose "Show in Table" * Select the filter and choose "Refresh" Watch the breakpoint being hit TWICE, by two different workers. Ssh runs into an exception because the ssh service is not re-entrant; I'm not sure if the dstore service is reantrant and prepared to handle multiple concurrent requests like this. Anyways, there should be only ONE remote resolve operation for a single filter being expanded.
By adding a synchronized on the filter reference adapter's internalGetChildren() method this seems to work okay. I checked in the change - can you see if this works for you too?
The synchronized seems not a good solution, for various reasons: * internalGetChildren() is a method that calls out to several other classes and methods, including some that may be contributed by extenders of RSE. Making such a "parent method" synchronized is _extremely dangerous_ since it locks the object against concurrent access from other threads... if user code calls out into the UI thread for instance (e.g. by IProgressMonitor.worked(), and the UI thread thinks it needs to access the filter reference adapter, a deadlock might occur. * Also, the fix does not solve the root problem. If I put a breakpoint in DStoreFileService.getFile(), and perform the refresh operation, then - If deferred queries are _on_, the breakpoint gets hit 2 times - If deferred queries are _off_, the breakpoint gets hit 4 times Thanks to the fix, the breakpoint hits are now serialized (one after the other), but this still means that choosing "refresh" leads to up to 4 queries into the remote system. This may be a performance problem on slow connections. Asking "Refresh" should refresh the underlying model only _once_ by getting new data from the remote system. Then, all the views should be updated by reading the internal model which has been refreshed already. I would strongly vote for removing the (synchronized) again because it is too dangerous, unless you can argue the fix is safe.
I found a bug that occurred in some cases of getParentPath() which would return the child path. With that fixed, I only see 1 getFile() call when deferred queries are one and 2 when they're off.
good! Do you think you could get rid of the 2nd query when deferred queries are off too? - If not this is probably not so much of an issue since I'd like the deferred queries to be the default anyways.
I've taken out the ISystemContainer.markStale() call that is in the system table provider flushCache() method. The actions and events should be taking care of the stale bit, not the table provider. The prevents the additional query.
[target cleanup] 1.0 M5 was the original target milestone for this bug
