Community
Participate
Working Groups
RSE should be a working EFS provider. * EFS provider: Problem is SchedulingRules -- need to move the cache outside the workspace? (bug 141806, bug 158773, bug 158777, bug 160492) * EFS browser: EFS contributions as RSE file services * Read-only flag handling in IFileService (bug 170832) Distinctions between RSE-files and EFS are: RSE is simpler, RSE is more target-fs-oriented (symlinks, permissions etc), RSE supports caching (EFS too?)
Created attachment 57890 [details] Error : create a remote C project by EFS filesystem click "File-->New project-->Standard C project -->Select File system: RSE -->browse path and select "RSE://102.123.123.121/home/ch/workspace/test" -->click "finish" button -->message error: project can not be created : Reason: invalid project description: ": is invalid character in resource name rse:" I want to know when this bug will be fixed?
Comment on attachment 57890 [details] Error : create a remote C project by EFS filesystem click "File-->New project-->Standard C project -->Select File system: RSE -->browse path and select "RSE://102.123.123.121/home/ch/workspace/test" -->click "finish" button -->message error: project can not be created : Reason: invalid project description: ": is invalid character in resource name rse:" I want to know when this bug will be fixed?
*** Bug 141806 has been marked as a duplicate of this bug. ***
*** Bug 158773 has been marked as a duplicate of this bug. ***
*** Bug 160492 has been marked as a duplicate of this bug. ***
Unfortunately the issue is not very easy to solve. The strategy which seems to be most promising right now, is to perform all EFS operations directly on the Service layer (IFileService / IHostFile objects) instead of using IRemoteFile. This would mean that EFS operations do not use the RSE cache but directly operate on the remote files (which might be an advantage, since the EFS API seems to be designed that way and doesn't support an explicit "Refresh" operation). Performance might suffer a little, e.g. when having multiple editors for multiple files. Note, though, that EFS has IFileStore.toLocalFile(EFS.CACHE) which explicitly requests a cached file. EFS brings its internal cache for supporting this, so there might not even be a need for using the RSE cache. A difference between the EFS cache and the RSE cache is, that the EFS cache is valid for a single session of Eclipse only whereas the RSE cache is persistent across multiple sessions and might thus be more applicable for replicating whole workspaces which are also used by some indexers. It looks like the single most important fix for making this possible is bug 162954: adding API for returning a Stream for download to IFileService, since EFS expects a Stream and it would otherwise be unclear where the file should be written to (if the RSE cache is not used). An alternative solution would be to do EFS the other way round: instead of the current FTP and SSH implementations of IFileService, implement these two as EFS providers and do an IFileService implementation based on an EFS provider. Given that we have the code for FTP and SSH already, converting them into EFS providers should be fairly simple. This approach would mean that there is not a single EFS provider of scheme "RSE" registered, but separate providers of scheme "FTP" and "SSH/Sftp". Adding new RSE IFileService providers for existing EFS implementations should be simple when we have that code. A dstore based EFS provider could be added later when streaming is supported for dstore. I think I'm slightly inclined to try this second option since it better componentizes the Target Management offering - people could use the plain FTP or SSH EFS providers without using RSE if they want. They cannot do this the other way round. RSE could still add persistent caching and exploring of non-workspace resources to the plain EFS offering; and when the scheduling/caching issues are finally resolved, RSE could even be an EFS provider of kind "rse" which uses the native EFS providers of kind "ftp" or "ssh" at the backend. One thing to find out for this second approach is, how the session management, password and passphrase management (i.e. the work that the RSE IConnectorService does) would be done for the EFS provider. With respect to timing, we are working on this with high priority but can't yet commit for anything other than what the targetmilestone says -- "2.0". Comments are very welcome!
*** Bug 173930 has been marked as a duplicate of this bug. ***
It looks like a proper change, which puts the RSE EFS Wrapper on top of IFileService / IHostFile instead of IRemoteFileSubSystem / IRemoteFile, would be along the following lines: * In RSEFileSystem.getStore(uri), it looks suspicious that there is a cache of (_fileStoreMap). I think I'd start testing without any cache and re-get fresh files every time. * In RSEFileSystem line 139, fs.getRemoteFileObject(path) seems to be the main reason for all problems, because this uses the RSE cache which needs the Eclipse Workspace and the Scheduling Rules. Instead, something like this should be better in order to use IHostFile: if(fs instanceof FileServiceSubSystem) { FileServiceSubSystem fss = (FileServiceSubSystem)fs; IFileService service = fss.getFileService(); IPath p = new Path(p); String name = p.lastSegment(); IPath parent = p.removeLastSegments(1); IHostFile hostFile = service.getFile(null, parent, name); return FileStoreConversionUtility.convert(null, hostFile); } * Instead of RSEFileStoreRemoteFileWrapper, there needs to be RSEFileStoreHostFileWrapper which takes an IHostFile in the constructor instead of an IRemoteFile. It may also need an instance of IFileService, and I'm not sure how it would deal with disconnect / reconnect. Getting the InputStream for EFS will be a lot easier when bug 162954 is fixed. * FileStoreConversionUtility needs to be changed to create RSEFileStoreHostFileWrapper objects instead of RSEFileStoreRemoteFileWrapper.
SSH should now work. I've still kept one file system provider, but with a SSH Only system type or SSH subsystem under other system types, user should be able to open a file, edit and save it. Restarting workbench should also work, i.e. if a file was opened in the previous session, it should be open on restart for editing. I'm seeing some problems with FTP. Project contents do not show. I do not think it's an EFS problem. Also, Dstore won't work since getOutputStream() and getInputStream() has not been implemented for the service. I have kept the implementation using IRemoteFile and RemoteFileSubSystem. However, this does not mean that we can not have different providers for SSH, FTP, Dstore and so on. We can keep the implementation using RemoteFileSubSystem so that it's generic and provide deifferent file systems.
(In reply to comment #9) > SSH should now work. That's great news! > Restarting workbench should also work It's great that you thought about that! > I'm seeing some problems with FTP. Project contents do not show. Could this be related to the SHOWHIDDEN Preference, i.e. not show "hidden" files which are starting with a dot ? > Also, Dstore won't work since getOutputStream() and > getInputStream() has not been implemented for the service. I don't quite understand how that relates to the next thing you say. If you still use IRemoteFile, you'll still have a cache, and you'd create a stream for the file from the cache... or are I totally misunderstanding something? > I have kept the implementation using IRemoteFile and RemoteFileSubSystem. > However, this does not mean that we can not have different providers for SSH, > FTP, Dstore and so on. I don't understand this... didn't we want to work on IFileService directly, instead of ISubSystem? And not use the cache? If we not use the cache, what do you need the IRemoteFile for? Why not just use IHostFile? I'm going to look at the code now...
Martin, using IRemoteFile does not mean we have to use the temp file cache. I bypass the temp file project and use the input and output stream from the file service to read/write. Datastore does not have the input and output stream implemented and hence does not work. We could change the impelementation to use IFileService and IHostFile instead. Right now, IRemoteFile is simply acting as a wrapper for a host file, to query various attributes. It is not being used to cache files in the remote systems temp project, as it was before. Caching and IRemoteFiles are not related. It's the RSE transfer classes that use the temp project. Note that this does not prevent us from being able to have different EFS file systems for SSH, FTP, etc if we so choose.
I have released all the fixes we've got right now for TM 2.0M5, but after code inspection it seems to me that it's not yet mature enough to be declared "official". For instance, * the handling of hidden files (SHOWHIDDEN constant) is definitely experimental right now, since it doesn't preserve a user's preference. * The "Create remote project" context-menu doesn't work reliably yet (Got an error: CoreException: This is a directory). * Setting up a new project (New > Project) with specifying RSE as the filesystem provider works when the selected directory is empty, but fails when there is contents in it including directories. * In the "browse" dialog which is available from "New > Project", it should be possible to create new folders. Summing up, the Experimental EFS Provider has improved a lot, and it's now possible to play with it over ssh connections, but there is still more work to do before it can be declared official.
Status as of TM 2.0M6: EFS integration is basically operational and testable, but blocking bug 181917 makes it impossible to work for an extended time because the SSH connection quickly runs out of file descriptors. This bug is fixed in HEAD and released for I-builds >= I20070410.
Work on the plan item is complete as per TM 2.0M6a, more work being done in the form of bug fixes now (tagged as [efs] in the subject).
[target cleanup] 2.0 M6 was the original target milestone for this bug
