Re: [ptp-dev] Remote Project

[Chris Recoskie <recoskie@xxxxxxxxxx>]

In my experience the two slowest parts of remote project creation are the following:

1.  The initial resource refresh.  There are improvements that both the Platform and RSE could make to help this be faster.  It would be best if the refresh happened in the background, incrementally, but right now it doesn't work that way, and there are lots of refresh requests that go over the wire for each resource instead of one single request for the whole tree.

2.  Creating the ICProjectDescription takes a long time regardless of project size.

Once the project is setup and the indexer has run, there is an overhead to doing remote lookups on the server but it's generally tolerable.

On source that big, you might also need to check the server side heap size for the JVM.  Depending on how big any one particular file is, you might need more heap to parse it.  Same goes for indexing locally for that matter.

===========================
Chris Recoskie
Team Lead
Rational Developer For AIX & Linux C/C++
Rational Developer For System z C/C++
IBM Eclipse CDT and RDT
IBM Toronto

Doug Schaefer ---06/24/2014 04:50:41 PM---This has been a very interesting discussion. It certainly reminds us why the remote project support

From: Doug Schaefer <dschaefer@xxxxxxx>
To: Parallel Tools Platform general developers <ptp-dev@xxxxxxxxxxx>
Date: 06/24/2014 04:50 PM
Subject: Re: [ptp-dev] Remote Project
Sent by: ptp-dev-bounces@xxxxxxxxxxx

---

This has been a very interesting discussion. It certainly reminds us why the remote project support was added in the first place. I personally think that it's such a hard problem, there probably isn't a good solution for hooking up an IDE to such an extremely large code base located on a remote machine, as much as we want there to be.

But one take away I have that might make synchronized projects work out is that you're desktop probably has as much power as the server you're running the indexer on, especially since the indexer is single threaded. It'll probably take the same amount of time to index on either and then you're left with the remote tooling doing expensive look ups on the server and the latency in passing results back to the workstation. I'd be excited to hear if there was much success using it that way and kill my doubt.

Also, Gigs of source isn't such a big thing any more when we have terabyte drives in our laptops. It may be an expensive first time configuration to get all that copied over to the workstation, but if the synchronizer has done it's job well, things should be very fast after that.

Anyway, just another opinion for the pile. You get to choose your path, but I'm afraid the remote project one isn't well travelled and I'm not as confident on where it leads.

Doug.

---

From: ptp-dev-bounces@xxxxxxxxxxx [ptp-dev-bounces@xxxxxxxxxxx] on behalf of Roland Schulz [roland@xxxxxxx]
Sent: Tuesday, June 24, 2014 4:41 PM
To: Parallel Tools Platform general developers
Subject: Re: [ptp-dev] Remote Project

I recommend to use synchronized project and tune the local indexer. See e.g. http://stackoverflow.com/questions/9565125/whats-the-recommended-eclipse-cdt-configuration-for-big-c-project-indexer-ta for tips of how to do that. 

Roland


On Tue, Jun 24, 2014 at 11:54 AM, Vishal Gupta <vishal.vit@xxxxxxxxx> wrote:

Ideally telling Yes i want to get the Indexing done on the server.  
And i am hopefull that it will ease the load on the local machine.

Yes i want to get everything index, but currently even my smaller project is not getting indexed


On Tue, Jun 24, 2014 at 9:11 PM, Roland Schulz <roland@xxxxxxx> wrote:
Hi,

do you want to use the index? Do you need everything indexed?

Roland


On Tue, Jun 24, 2014 at 8:22 AM, Vishal Gupta <vishal.vit@xxxxxxxxx> wrote:
Hello Greg,

Thanks for the feedback.

Let me give you a background why i thought of giving a try to the remote project.

1. I have a very huge code base ( in GBs)
2. Using Eclipse to import the complete code base on the local machine is crazy. Eclipse can't withstand such a huge code base and the indexer makes things non-usable.
3. So the solution was to checkout the code base on server and I tried to use the Remote Project to access it. The benefit i thought was that the indexing (which is the most memory intensive operation will happen on the server).

Now as you are suggesting to use the Synchronized project, my biggest nightmare "indexing" will again be happening on the local machine. Which will again make eclipse non-usable for my code base. Another thing is to sync the code base initially, my personal feeling is that it will take ages to do initial syncing between geographic different location.

I am very much open to  contribute code if you people can guide me to a solution.

Thanks,
Vishal



On Tue, Jun 24, 2014 at 5:16 PM, Greg Watson <g.watson@xxxxxxxxxxxx> wrote:
Hi Vishal,

Many of the problems you describe are also apparent even when accessing a remote project in the same geographic location, or even the same local network, and tend to be inherent in the nature of remote projects. It may be possible to speed up directory listings as you suggest, but this will not do much to speed up the overall performance of remote projects.

As a result of these issues, we have transitioned to another type of project called synchronized projects. These projects maintain both a local and remote copy of the source files. Apart from the time it takes to do the initial synchronization, these projects tend to work much faster with Eclipse, since the indexer and other expensive operations only happen on the local copy of the files.

Synchronized projects are available in both the Kepler and soon-to-be-released Luna versions of Eclipse for Parallel Application Developers package.

Ongoing development of remote projects has pretty much ceased, so it is unlikely there will be any future updates unless you would like to contribute something.

Regards,
Greg

On Jun 24, 2014, at 6:59 AM, Vishal Gupta wrote:

> Hello ,
>
> I have just started using Remote Project.
> I am accessing my project in a geographically different location. (Remote machine the server is in USA and local machine is in India)
> My initial impressing is that it's very slow to access even a small
> project with 1000 files (around few MB). I am using the "remote tool" configuration
> to access my project on remote machine.
> I tried looking into it and my initial finding is :
>
> Project Creation itself take a long time as the .project file and .setting folder is created and updated on the remote machine. Could it be possible to create the .project on the local machine ?
> I think the same happens with RSE connections  where a hidden project is created on the local system for the connections and the configuration files are stored in it.There may be better solution for sure.
>
> For me the refresh of the files and folder is also very slow, my initial feeling is that the "ls" operation via sftp connection is taking more time. Any possibility to speed it up?
>
> The Indexing for my CProject is not working. The PDOM files is getting created on the remote system but the indexing is not happening. Do i need to do anything extra ?
> I checked the status of the Remote Indexing operation and it only had info and no error.
> I am still digging into the problem. Just wanted to check if i have missed anything ?
>
>
> Thanks,
> Vishal
> _______________________________________________
> ptp-dev mailing list
> ptp-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/ptp-dev

_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/ptp-dev



--
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309

_______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/ptp-dev

-- 
ORNL/UT Center for Molecular Biophysics cmb.ornl.gov
865-241-1537, ORNL PO BOX 2008 MS6309 _______________________________________________
ptp-dev mailing list
ptp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/ptp-dev