Bug 453826 - Synchronized projects should properly handle files used for version control
Summary: Synchronized projects should properly handle files used for version control
Status: NEW
Alias: None
Product: PTP
Classification: Tools
Component: RDT.sync (show other bugs)
Version: 8.1   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-12-01 16:30 EST by Anthony Ette CLA
Modified: 2014-12-30 15:20 EST (History)
2 users (show)

See Also:


Attachments
Screenshot showing erroneous local revision (153.95 KB, image/png)
2014-12-02 11:17 EST, Anthony Ette CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Anthony Ette CLA 2014-12-01 16:30:14 EST
Overview:
I'm running Eclipse for Parallel Application Developers, version: Luna SR1 (4.4.1), build id: 20140925-1800.

I've followed the steps to setup a synchronized project that uses CVS and everything seems to be working great until I do a build on the remote system (Linux EPEL5).  When I build on the remote system then sync the changes back, too many outgoing changes are being marked in the Project Explorer view.  However, the Team->Synchronize view shows the correct number of outgoing changes.  I suspect this is a strange interplay between Git and CVS since Git is used to handle project sync.  More details are provided below.

Steps to Reproduce:
1) Create new project from CVS
2) Convert to Synchronized project
3) Build (or just touch) a file on the remote system (Linux EPEL5 if that matters). Note, if important, that I'm currently doing this via Cygwin ssh connection.
4) Sync Active Now (assuming "active" is the remote system)
5) Ensure that the Team->Synchronize view is also refreshed
5) Compare the Project Explorer view to the Team Synchronize view and note that the Project Explorer view is indicating too many outgoing changes.

Actual Results:
Too many resources are marked as outgoing changes in the Project Explorer view.

Expected Results:
Only the file(s) that were modified by build (or touched) should show as outgoing changes in the Project Explorer view.  This is exactly what you see in the Team->Synchronize view.

Additional information:
I added the local .ptp-sync Git repository to the Git Repositories view in an attempt to gather more information about what is going on.  However, this raised more questions than answers as I found many more git commits going on behind the scenes than I expected to see.  For example, Git is controlling all the CVS configuration files (e.g. */CVS/Entries) and is attempting to merge them when sync'ing between systems (this seems wrong!).

What I think is happening:
I think the Git repository is being used to populate the Project Explorer view, when I really expect the CVS repository (as shown in the Team->Synchronize view) to populate this view.  I think this is the desired behavior because the fact that PTP uses Git for sync is supposed to be "invisible" to the user.  Although I could continue to work exclusively with the Team->Synchronize view (which is working as expected), I would like to be able to make better use of the Project Explorer view (even if it means becoming more familiar with the nuances of the Git sync implementation).

Thank you in advance for your time developing and maintaining this great product!
Comment 1 Anthony Ette CLA 2014-12-02 00:30:18 EST
In digging deeper, I uncovered the issue is actually brought on by running cvs commands on the remote system (not by performing a build).  Please modify the Step to Reproduce with this new finding:

1) Create new project from CVS (use existing module/members)
2) Convert to Synchronized project
3) Run cvs command on remote (Linux EPEL5) system (e.g. 'cvs -Q stat <file>')
4) Sync Active Now (assuming "active" is the remote system). Here you can see the effect of the cvs stat command if you check the Git sync repo.  The timestamp for <file> was updated in his corresponding CVS/Entries file.  This seems incorrect as the only thing that ever touch a CVS internal file is a native CVS commit/add/update.
5) Ensure that the Team->Synchronize view is also refreshed
6) Compare the Project Explorer view to the Team Synchronize view and note that the Project Explorer view is indicating too many outgoing changes.  Project Explorer view shows outgoing change on <file> whereas Team Synchronize view does not.
Comment 2 Anthony Ette CLA 2014-12-02 11:17:53 EST
Created attachment 249101 [details]
Screenshot showing erroneous local revision
Comment 3 Anthony Ette CLA 2014-12-02 11:18:04 EST
More information:
When you run the 'cvs stat <file>' command from a CVS-aware directory, the timestamp on the CVS/Entries file in the directory where <file> resides is changed but the content remains the same (since cvs stat is a "read" not "write" operation on the repository).  However, Eclipse is making the assumption that if the timestamp on the Entries file changes, the content needs to be updated (even though it doesn't in the case of cvs stat or any other "write" operation for that matter).  When Eclipse modifies the CVS/Entries file, <file> is shown as an outgoing change in the Project Explorer view even though nothing has changed in the file!  The attached screenshot exhibits the behavior more clearly where <file>=arincfiles.c.  Note the erroneous local revision of the file highlighted in the History view as a result.

So, I guess to add the most succinct description of the bug and the desired fix now that all the facts are in: 
Eclipse PTP should treat CVS internal files (e.g. CVS/Entries) unique from any other types of files and, specifically, it should check their contents (not just timestamp) to determine when changes have been made and a sync is required.  This way, cvs repo "read" actions such as 'cvs stat' will not have undesired affects on the PTP Synchronized Project.

Thank you all again for your time and consideration.
Comment 4 Roland Schulz CLA 2014-12-02 12:57:41 EST
Have you tried to add the /CVS/ folder to the sync filter?
Comment 5 Anthony Ette CLA 2014-12-02 14:32:33 EST
(In reply to Roland Schulz from comment #4)
> Have you tried to add the /CVS/ folder to the sync filter?

Are you talking about the one under Remote Development->Synchronized Projects->File Filtering or the one under Team->CVS->Ignored Resources?  Is the distinction between the two btw that the first one prevents the filters from being sync'ed (and subsequently from being controlled by Git) and the second will allow them to be sync'ed (and hence controlled by Git) but it will not try control them in CVS (i.e. you won't see them in the Team->Synchronize view)?  This was a point of confusion for me even after reading the docs so thank you in advance for clearing this up for me.

Nonetheless, I don't think your suggestion will work because the CVS directories are needed on the remote machine in order to issue cvs commands such as 'cvs stat'.  It may work if you could create a way to only sync the CVS dirs the first time or something like that...

Also, I discovered that CVS actually does modify the Entries file during 'cvs stat' if the filesystem timestamp doesn't match what's currently in there.  Because of this, I don't believe there's any good way for Eclipse (or any other system/method) to allow for cvs commands on the remote machine (with the sole exception of something like what I suggest above about syncing CVS dirs first time only).  Thoughts?  Should we give up and just say "don't use cvs on the remote copy" or can it be made to work?
Comment 6 Roland Schulz CLA 2014-12-02 15:55:37 EST
Yes your understanding of the two types of filters is correct.

If you don't need the two CVS repositories to be synced, then you can do an initial sync without a sync-filter (which copies the CVS folder to the remote) and then add the /CVS/ sync-filter (which prevents it from syncing it from then on). When adding a sync filter the filtered files/folder are not removed again. Notice that if you do this, the CVS log will not be updated on the remote site when you do a "cvs up" on the local site. If you want to have an up-to-date CVS on the remote you have to do the "cvs up" there too.

If you run the cvs command which causes the issue on the remote side, locally on the command line from outside of Eclipse, do you get the same behavior?
Comment 7 Anthony Ette CLA 2014-12-02 16:17:45 EST
Ok good thank you.

Well technically, there's only 1 CVS repository and it would be nice I guess if we could integrate with that repository seamlessly on both sides.  Although the solution we suggest below about synching the CVS dirs first time only would then allow you to use cvs commands on the remote machine, you now MUST use cvs commands on the remote machine (e.g. to perform updates as you pointed out).  This sounds like more of a hinderence than a benefit.

I will check the behavior when I use cvs commands locally; if the behavior is the same then it's probably something Eclipse is doing and/or could do better in managing CVS info, if not then I think the problem we're identifying is that CVS (specifically how it uses filesystem timestamps) precludes us from coming up with a workable cross-filesystem solution.  Basically, in the latter case, no CVS commands should be issued on the remote machine.  If this turns out to be true, it would be ideal if the outcome of this bug were for Eclipse to add a friendly reminder in the docs that one should not be attempting to use cvs on the remote machine so people can avoid the frustration I'm experiencing now!

Thanks again; I'll report back later tonight or early tomorrow.
Comment 8 Anthony Ette CLA 2014-12-02 16:43:28 EST
Ok so cvs stat is hanging in my local setup.  Probably because I don't have Cygwin configured to properly communicate with the remote repository (now it's getting circular because this is why I switched to Eclipse in the first place lol!).  I did notice however that all the local filesystem timestamps match the local CVS/Entries files which means I wouldn't expect to see the same behavior if I had everything setup properly to actually test it...can you easily check on your end?  

Either way I think it's a reasonable assumption that, if you've installed Eclipse PTP, it is becuase you don't want to deal with issuing CVS commands through a remote terminal window.  If you still want to do that, you can, just not from the Eclipse sync'ed dir.  In other words, I could do all my CVS work locally then go over to the Linux EPEL5 box and checkout from the repository to another location (not the Eclipse sync'ed dir).  Now I can again relish in the joy of CVS command line on the remote machine.  Do you agree that this would be an acceptable workflow?  If so, how much trouble would it be to add some notes about this to the Eclipse docs so people don't shoot themselves in the foot like I have?
Comment 9 John Eblen CLA 2014-12-03 15:08:46 EST
Synchronized projects are meant to support the development of remote code from within Eclipse, and thus they operate independently of version control. So the best approach, I think, is to isolate version control to either the local side or the remote side, and then use filtering to prevent transferring version control files.

If you want the version control to be on the remote side, simply create a sync project to the remote directory. In this case you would not be using Eclipse's CVS support. For such support, you would need to have local version control. Either way, be sure to add the CVS directory to the filter from inside the new project wizard so that those files will never be sync'ed. Remote CVS commands should then work fine, if version control is remote, because there will be no interference with the files for version control.

If you need to run remote CVS commands (or other remote commands) occasionally, the new GUI terminal is ideal for this (right-click on project and select "Show Terminal").
Comment 10 Anthony Ette CLA 2014-12-03 15:18:34 EST
I agree completely with your assessment and recommendation, John.  My main problem was that this important design "feature" if you will is not clarified in any of the docs that I read.  Can you point to something that indicates you shouldn't be attempting to interact with version control on boths sides of a Syncrhonized Project?  If not, do you think such a note would fit somewhere within the current help (perhaps the PTP SYNC Project FAQ at https://wiki.eclipse.org/PTP/sync-projects)?
Comment 11 John Eblen CLA 2014-12-03 16:10:47 EST
You are correct. We don't currently have this information in our documentation, and it should be there. I have added it to the FAQ that you mentioned. We can also add it to the user guide for the next release.
Comment 12 Anthony Ette CLA 2014-12-03 23:00:35 EST
Perfect; I'm happy with this outcome and the additional section in the FAQ is helpful.  I guess feel free to mark as resolved/closed/whatever as you see fit.  Thank you guys again for developing and maintaining this great product!
Comment 13 John Eblen CLA 2014-12-30 14:16:53 EST
Thank you for your feedback Anthony. I've changed this bug to a feature request, because synchronized projects could possibly be enhanced to detect and handle version control files correctly. This would probably require special code for each system (CVS, SVN, Git, etc.). I'm not sure how feasible it is, but it is worth considering.
Comment 14 Anthony Ette CLA 2014-12-30 14:51:21 EST
John,

Thank you for keeping this alive.  I don't know how much trouble it would be to support - but it would certainly be useful if sync projects could handle this!

That said, I've now found another issue I don't think is specifically related to PTP or sync projects: Eclipse does not seem to be able to connect to a local CVS repository.  I did find a section in the help pages that describes a way of "hacking" around this limitation, but it doesn't seem to work correctly in all cases.  The "hack" is described at 

http://www.dvteclipse.com/documentation/e/How_can_I_configure_Eclipse_to_use_a_local_CVS_repository.3F.html

Steps to recreate:

1) Install latest PTP RPM on RHEL 5
2) Configure Eclipse to use local cvs repo (see http://www.dvteclipse.com/documentation/e/How_can_I_configure_Eclipse_to_use_a_local_CVS_repository.3F.html)
3) Create a new project from existing CVS module
4) Note that eclipse can read/write to repository without problems but other clients (including command line) will fail because they can't find the repository. This must be a flaw in how Eclipse handled local CVS repo and/or the instructions in the URL above (maybe they're incomplete or I did not follow them correctly). Note that I even did the part that says: "In order to make cvs work in the command line as well (with projects checked out from Eclipse) you have to set the system variable CVS_RSH to point to the script and CVSROOT to the the local absolute path to your repository."  So, despite the article saying it can work for other clients outside Eclipse, it just doesn't seem to...

Because I don't believe this is related to PTP, please help me make sure this behavior I'm now describing gets to the right place in the system (this may be a new bug in the general Eclipse location in bugzilla?).

Thanks again for all your time and support.
Comment 15 John Eblen CLA 2014-12-30 15:20:23 EST
Yes, this falls outside of PTP. I recommend contacting the Eclipse CVS team:

https://www.eclipse.org/eclipse/platform-cvs/

This site contains the developers' mailing list address.

Bugs can be filed under Eclipse/Platform/CVS on bugzilla.