Re: [geclipse-dev] File Transfer Service / 3rd Party Transfers

[Szymon Mueller <mael@xxxxxxxxxxxxxxxxxx>]

Hi all,

Continuing last post by Mateusz (from 03.09.2008) let me describe transfer 
mechanism, and especially transfer resuming functionality.

1. Transfer manager uses a repository (xml) file, stored in gEclipse 
metadata, containing informations about pending transfers. This 
information will be used to restore the transfer in case of something 
unexpected happens (gEclipse crashed/is closed while transfering etc.).
2. When user starts new transfer new entry is created in repository file.
3. After creating new entry in the repository file, third party transfer 
extension point is queried if there is any extension which can handle this 
transfer (i.e. manage transfer from source_scheme to destination_scheme). 
If no such extension is found default transfer service is used. Default 
transfer service transfers data using local machine: source -> local -> 
destination. After choosing best transfer service, the transfer operation 
starts.
4. Transfer service can set additional informations (transfer token id 
etc.) and save them in the repository file.
5. After transfer completes (it doesn't matter if it was successful or 
not) transfer entry is removed from the repository.

At gEclipse startup the repository file is checked if there are any 
pending transfers. A dialog is shown which lists all unfinished transfers. 
User then chooses which (if any) he wants to resume. Again extension is 
queried for best transfer service and transfer is resumed. In case of the 
default transfer service this means that the operation is run from the 
begining. For other transfer services this could mean resuming transfer 
from last checkpoint, or just checking if the transfer has already 
finished (fire-and-forget 3rd party transfers).

Regarding the technical stuff (still a lot of work to do here):
> Concerning the technical side I am curious how this will be implemented
> on the MW-independent side. I guess this has to be covered by our UI's
> transfer operation? Or will you provide a completely new mechanism here?

Currently I am still using GridElementTransferOperation class, but some of 
the functionalities (like the fundamental ones: copyFile and copyFolder) 
are in fact moved to different classes. I think that in later development 
this class will be removed.

> Obviously the transfers should be implemented in the
> EFS-implementation's copy/move-methods. Are these already used by our
> transfer operation? I do not know the current implementation but the
> initial implementation made use of the IResource methods for copying
> files. So this may also be a subject of change then.

Current implementation of the transfer operation (and the default transfer 
service from the above description) is just a simple stream transfer.
Regarding the first part: copy/move methods of the EFS is not a good place 
for the transport operation: we may have different source-scheme and 
target-scheme (e.g. srm -> gridftp), so these operation should be defined 
outside of the IFileStore implementation. Currently simple interface is 
definied:

public interface ITransferService {
  public boolean transfer( final IFileStore source,
                           final IFileStore destination,
                           final IProgressMonitor monitor );
}


On the teleconference tomorrow i will show the first implementation of 
this functionality and hopefully hear your opinions and remarks.

Cheers,
Szymon