RE: [dsdp-tm-dev] Evaluation of RSE for Remote Server Support in PTP

[Tianchao Li <lit@xxxxxxxxx>]

On Tue, 2006-05-16 at 15:09 +0200, Oberhuber, Martin wrote:
> Hello Tianchao,
> 
> I finally managed to read through your document. I think that
> quite a few things have changed already since the original 
> writing. So here are a few comments from my side:
> 
> * Replication.
>   In your 3 remote scenarios (local, remote, hybrid), I'm
>   somewhat missing the "replicated" scenario which is assumed
>   by many as the most common one
>   (See https://bugs.eclipse.org/bugs/show_bug.cgi?id=106176#c33)
>   Perhaps your "local" scenario means exactly this, but it 
>   became not quite evident from the document. Note that repliation
>   can be done through a variety of means (including simple 
>   rsync on a commandline, which can be fine tuned e.g. not to
>   sync remote *.o files which are not needed locally). Perhaps
>   your headline should simply read "Local Project, repliated".
>   In the Roadmap, your points 3 and 4 (project export, project
>   update") seem to go in this direction.

"replicated" was not actually mentioned explicitly as a scenario in the
original document. Even if it is, it is simply another categorization of
the same thing, from the perspective of actions. I would like to
categorize however according to the placement of the workspace (or the
Eclipse machine) and the files. I can't see which categorization is
better than the other.
Based on the categorization of placement(Local, Remote, Hybrid), you can
have different actions to achieve what you want. And replication is one
of them, that can be applied to Local and Hybrid scenario.

> * Static Analysis.
>   In your workflow list (code edit, remote build, debug etc)
>   I was somewhat missing the "source analysis" step. Note
>   that source parsers like the ones in JDT and CDT need to
>   analyse the entire workspace.

That is one advantage of using the EFS. If JDT and CDT are enabled with
EFS, the access to all the sources in the workspace are through the EFS
API. And the locality of the files are transparent to JDT and CDT.

>   When you use a remote filesystem like EFS, each and every
>   file needs to be transferred to local for this analysis 
>   step. "stat" calls need to be made for each and every file
>   very often. The two ways around this are
>     a) Allow a source parser to run remotely (I heard rumors
>        that IBM is thinking about remote C/C++ parsing)
>     b) Repliate the workspace efficiently and parse locally.

Everything has pros and cons. But if you want static analysis with ease,
I mean, without modifying the existing structures a lot, you need to
accept it. However, EFS has local caches and you only need to transfer
the 'remotely'  file for only once. Remote parsing is fine, and can be
considered as another enhancement. But it does not prevent us to use
EFS.

> * RSE Status.
>   - Yes we do intend to collaborate closely with the Platform
>     and make services available in a manner as general as 
>     possible.
>   - EFS integration has been improved, it is now possible to
>     create projects on an remote EFS provided through RSE.
>     This is still experimental though (not sure if it will
>     make it into M2), your help here would be appreciated.
I've seen this progress. Actually I'm not saying that RSE has no value
at all in providing remote file access. It is good that you can provide
EFS for the remote machine, and actually I'm very willing to use such
remote file systems (though I'm meeting some bugs, ref. bug #143023).
Merely integration with EFS is however not the best solution. There are
a lot overlapping between the File subsystem and the EFS. I find no
reason why you can't use EFS API to replace your file subsystem.

>   - Remote Search.
>     I do not think that remote search actually duplicates
>     current functionality, since the big plus of it is that
>     all file accesses are remote with no actual data transfer.
>     You cannot have this through EFS (every searched file
>     is transferred to local).
You are right. What I actually meant is that we can achieve the
samething with EFS, but of course at a little more cost. However, we
automatically has all the advantage of the Eclipse - we can apply any
type of search on the remote project (can you do Java search with remote
search?)

>   - Datastore.
>     Note that a remote datastore server can also be launched
>     through an ssh channel for a single user only. So your
>     point 1 "requires a service on the remote" is not true.
>     The Dstore Daemon is just ONE of many launching options
>     (though the most simple one). All you need is a 1.4 
>     compatible JVM on the remote system.
>     Datastore also supports SSL encoded transfer.

I haven't tried datastore up to now. But anyway, you need to start a
custom server on the remote machine.

>   - As of now, ssh and sftp support has been added to the RSE.
>   - Refactorings are underway to better separate UI and non-UI
>     parts of the RSE. Do you have specific needs for RCP?

> We'd really like to collaborate more with the you, and the
> PTP project. RSE can, for instance, provide the shell channel 
> through which remote build/debug can be started in a pluggable
> manner.

That is great enhancement. I have actually already tried to use RSE
shell in PTP. The separation of UI and non-UI parts is definitely
something we want. We have a plan to use RCP in PTP.

> Regarding remote file support, I guess the big question is
> what setup you envision. Local replicated workspace (then 
> you don't need remote search), Or a remote workspace (then
> RSE Filesupport would probably make most sense for you).
> In any case, RSE shell / command support would make sense
> for you, I guess. With the ability to contribute RSE 
> subsystems specific for remote parallel machines and their
> configurations in the future, share those in a team, visualize
> their status etc.
> 
Sure. We'd like to collaborate and contribute to RSE. RSE is good, but
needs improvement. 

> I have also uploaded these comments to your Wiki document,
> on the "discussion" tab:
> http://wiki.eclipse.org/index.php/Talk:PTP/designs/remote.
> 
> Cheers,
> Martin
> --
> Martin Oberhuber - WindRiver, Austria
> +43(662)457915-85
>  
> 
> > -----Original Message-----
> > From: dsdp-tm-dev-bounces@xxxxxxxxxxx 
> > [mailto:dsdp-tm-dev-bounces@xxxxxxxxxxx] On Behalf Of Tianchao Li
> > Sent: Tuesday, April 18, 2006 1:11 AM
> > To: dsdp-tm-dev@xxxxxxxxxxx
> > Subject: [dsdp-tm-dev] Evaluation of RSE for Remote Server 
> > Support in PTP
> > 
> > Hello,
> > 
> > I have recently accepted the task of providing remote server 
> > support in 
> > the PTP project. As a first step, we have done a survey of 
> > related tasks 
> > and evaluated the RSE and new alternate file system support 
> > from Eclipse 
> > core intensively. The document is now available at 
> > http://wiki.eclipse.org/index.php/PTP/designs/remote.
> > We believe that the RSE needs a lot effort to meet the need of target 
> > management and to catch up with the recent progress in the alternate 
> > file system support from Eclipse core. We would like to 
> > collaborate with 
> > all related parties on RSE.
> > 
> > Regards,
> > Tianchao Li
> > 
> > ===========================
> > Tianchao Li
> > Technische Universitaet Muenchen
> > Institut fuer Informatik, I10
> > Boltzmannstr. 3
> > D-85748 Garching
> > Room 01.06.061
> > Phone: +49 (89) 289 18477
> > Fax:   +49 (89) 289 17662
> > mailto:lit@xxxxxxxxx
> > http://www.lrr.in.tum.de/~lit
> > 
> > 
> > 
> > _______________________________________________
> > dsdp-tm-dev mailing list
> > dsdp-tm-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/dsdp-tm-dev
> > 
> _______________________________________________
> dsdp-tm-dev mailing list
> dsdp-tm-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dsdp-tm-dev
>