Bug 194204 - [ftp] Renaming Files/Folders moves them sometimes
Summary: [ftp] Renaming Files/Folders moves them sometimes
Status: CLOSED FIXED
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: 2.0   Edit
Hardware: PC Windows XP
: P2 critical (vote)
Target Milestone: 2.0.1   Edit
Assignee: Javier Montalvo Orús CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords:
: 194207 194999 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-06-25 10:00 EDT by Kevin Doyle CLA
Modified: 2012-05-23 17:17 EDT (History)
4 users (show)

See Also:


Attachments
patch (1.61 KB, patch)
2007-07-02 04:30 EDT, Javier Montalvo Orús CLA
mober.at+eclipse: iplog-
Details | Diff
Patched Jar (32.48 KB, application/java-archive)
2007-07-03 09:53 EDT, Martin Oberhuber CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Doyle CLA 2007-06-25 10:00:07 EDT
After renaming a file/folder in some cases the file/folder is renamed and MOVES.

Ways I can reproduce the file moving:
1. Open the file in an editor.
2. Rename the file.  Everything looks okay, but if you refresh the parent it disappears and should be under the folders parent.

Ways to reproduce the folder moving:
Hard to get this to happen.  I can't seem to get it to happen directly under the My Home filter, so it might be best to try a few levels deep under the My Home filter.  Expand the directory you are going to rename and rename it.

-----------Enter bugs above this line-----------
TM 2.0RC4 Testing
installation : eclipse-SDK-3.3RC4
RSE install  : RSE 2.0 RC4
java.runtime : Sun 1.5.0_11-b03
os.name:     : Windows XP, Service Pack 2
------------------------------------------------
Comment 1 David Dykstal CLA 2007-06-27 15:04:05 EDT
At Kevin's request I was able to verify that this happens by renaming a folder two levels deep under a filter. I had to rename it serially since this didn't happen the first time, but did the second. The console seems to show that we are issuing the correct commands.

I attempted to reverify and found that I could not reproduce the problem even after renaming the folder 12 times.
Comment 2 Martin Oberhuber CLA 2007-06-28 05:07:44 EDT
Dave - I'd like to know two things:

1. Console seemed to show the right commands, could you verify that 
     1. In reality the file is in the right place
     2. Locations below RemoteSystemsTempFiles where the file is?
   This would mean that the issue is a  SystemView display issue while the 
   right commands were executed, and it would not be a major issue. In that
   case, it would be likely to happen not only with FTP but also with dstore
   or ssh. The fact that it appears an editor needs to be involved seems to
   indicate that some Rename events are sent wrongly; tracing rename events
   either with the Event Log Console from EclipseCon, or by setting a 
   breakpoint in the Rename Event Creation code could help tracking these down.

2. Could you try reproducing this with ssh or dstore? - If not reproducable with
   those, it might be an FTP-only thing, perhaps even a bug in the FTP server
   or an issue with timing of the RNFR / RNTO commands. Javier have you heard
   of any such issue?

Tentatively assigning DaveD since you've had some exposure to this already and my feeling is that it's more event related than an actual FTP/Protocol thing. Setting Severity NORMAL since very hard to reproduce and priority P2.

Kevin and DaveD please give more info on your configuration, i.e. what FTP server you used and answer questions (1) above if you can reproduce it again.
Comment 3 Kevin Doyle CLA 2007-06-28 14:43:01 EDT
I repeated the steps for making the file move.

Console shows this:

RNFR /home/tester/RSE2.0-Testing/subfolder2/test2
350 RNFR accepted - file exists, ready for destination

RNTO test2---kevin
250 File successfully renamed or moved

In the RemoteSystemsTempFiles files the file is in the right place and the file is renamed correctly.  However on the server the file is not in the correct place.  The server I just reproduced this on was running Pure FTP.  Also had this happen on vsFTPd 1.1.0.

I've tried to reproduce this on ssh and dstore and haven't been able too.  It's ftp specific as noted in the summary.
Comment 4 David Dykstal CLA 2007-06-28 17:07:36 EDT
Thanks for the analysis Kevin!
Assigning back to Javier for further work.
Comment 5 Martin Oberhuber CLA 2007-06-29 05:46:00 EDT
Javier - could it be that a "cd" command is missing here, or that "RNTO" requires an absolute path just like RNFR?

I'm wondering why this happened only sometimes and not consistently. Marking as Major since the server does not look like what people expect it to look like - potentially making things not work properly on the server without telling the user so. This will be an item for the release notes.

Please look at this as soon as possible.
Comment 6 Javier Montalvo Orús CLA 2007-06-29 07:29:28 EDT
(In reply to comment #5)
> Javier - could it be that a "cd" command is missing here, or that "RNTO"
> requires an absolute path just like RNFR?

Yes, it is exactly the problem. The renamed file is placed in the current working directory (cwd), so if before renaming a file/folder another folder is expanded the cwd gets assigned to that folder and the renamed file gets moved there.


Comment 7 Martin Oberhuber CLA 2007-06-29 10:23:18 EDT
So, would this be a proper Workaround: before renaming the item, refresh and expand the folder in which the item resides?
Comment 8 Kevin Doyle CLA 2007-06-29 11:23:52 EDT
hmm.. I don't think refreshing the parent will work as a workaround for this.

When you open a file it changes the cwd to /.   Now if I refresh the parent of the file the console log shows this:

CWD /public_html
250 OK. Current directory is /public_html

PORT 9,26,148,193,8,115
200 PORT command successful

LIST
150 Connecting to port 62756

226-ASCII

226-Options: -a -l 

226 50 matches total

NOOP
200 Zzz...

CWD /
250 OK. Current directory is /

PORT 9,26,148,193,8,116
200 PORT command successful

LIST
150 Connecting to port 62801

226-ASCII

226-Options: -a -l 

226 36 matches total

NOOP
200 Zzz...

CWD /public_html/B2aaa
250 OK. Current directory is /public_html/B2aaa

PORT 9,26,148,193,8,117
200 PORT command successful

LIST
150 Connecting to port 62829

226-ASCII

226-Options: -a -l 

226 6 matches total

NOOP
200 Zzz...

CWD /public_html
250 OK. Current directory is /public_html

PORT 9,26,148,193,8,118
200 PORT command successful

LIST
150 Connecting to port 62878

226-ASCII

226-Options: -a -l 

226 50 matches total

NOOP
200 Zzz...

CWD /
250 OK. Current directory is /

PORT 9,26,148,193,8,119
200 PORT command successful

LIST
150 Connecting to port 62913

226-ASCII

226-Options: -a -l 

226 36 matches total

"/public_html/B2aaa" is the directory I did the refresh on.  The cwd is back to / after doing the refresh so the file is going to be moved when I do a rename.
Comment 9 Martin Oberhuber CLA 2007-06-29 11:35:49 EDT
Hm, this seems to be another indication that we are refreshing too much (bug #188160 which was fixed but should perhaps be reinstated as a new bug).

What if you press refresh (F5) on the file you want to rename, just before renaming it?
Comment 10 Kevin Doyle CLA 2007-06-29 13:06:13 EDT
I agree Martin.  At least on ftp it looks like we are requesting the list of files from folders we don't need for that operation.  Do you want to open a new bug for this or should I?

Refreshing the file itself doesn't work either.  This is my console log which I cleared just before I pressed F5 on the file.  File is in directory "/public_html/B2aaa".

NOOP
200 Zzz...

CWD /public_html
250 OK. Current directory is /public_html

PORT 9,26,148,193,10,30
200 PORT command successful

LIST
150 Connecting to port 46691

226-ASCII

226-Options: -a -l 

226 50 matches total

NOOP
200 Zzz...

CWD /public_html/B2aaa
250 OK. Current directory is /public_html/B2aaa

PORT 9,26,148,193,10,31
200 PORT command successful

LIST
150 Connecting to port 46725

226-ASCII

226-Options: -a -l 

226 6 matches total

NOOP
200 Zzz...

CWD /public_html
250 OK. Current directory is /public_html

PORT 9,26,148,193,10,32
200 PORT command successful

LIST
150 Connecting to port 46778

226-ASCII

226-Options: -a -l 

226 50 matches total

NOOP
200 Zzz...

CWD /
250 OK. Current directory is /

PORT 9,26,148,193,10,33
200 PORT command successful

LIST
150 Connecting to port 46807

226-ASCII

226-Options: -a -l 

226 38 matches total
Comment 11 Kevin Doyle CLA 2007-06-29 13:18:14 EDT
Actually, if I refresh a file twice it's cwd is the proper one after.  Not sure why the cwd is different if you refresh twice.  It might not happen always.  After doing that it seems opening files keeps the proper cwd most of the time.  Sometimes it shows the wrong cwd, but I'm having a much harder time reproducing it, where before I could 100%.
Comment 12 Javier Montalvo Orús CLA 2007-06-29 13:27:19 EDT
In my system the last CWD command always points to the refreshed or expanded folder. 
Which server are you testing against ? Martin, do you have the same behaviour ?
Comment 13 Kevin Doyle CLA 2007-06-29 13:56:16 EDT
Javier I'm using VSFTP and Pure-FTP.  Does it happen for you if you go a few levels deep in the ftp?

This seems to always show the wrong cwd for me:
Expand a folder.  Refresh that folder.  Refresh the parents folder.  Does it still show the correct cwd?

I tried that on 3 different ftp servers and the cwd was / or /username.
Comment 14 Kushal Munir CLA 2007-06-29 15:15:24 EDT
I see exactly the same problem that Kevin is reporting. The first refresh seems to be ok, but refreshing the parent sometimes will change the CWD to '/'.
Comment 15 Martin Oberhuber CLA 2007-06-29 15:32:33 EDT
I think the odd CWD may be due to the extra (unwanted) LIST commands. We've had cases where refreshing a directory also got status of its parents etc. - this would also explain why refreshing twice works -- first time the parent, grandparent etc. are all put into the RSE dircache so the 2nd time the status is no longer required from grandparent etc.

So I think the right things to do are:
1. Advertise workaround of refreshing twice (I just did that).
2. Fix the bug by locking the FTP session with semaphore during the rename,
   and performing the proper CD command before the RNTO.
3. File a new bug for the performance issue, i.e. [performance] RSE performs
   unnecessary remote list commands, then investigate and fix that one.

I don't think this depends on the FTP server, and I don't plan to try reproducing this myself.
Comment 16 Kevin Doyle CLA 2007-06-30 14:39:40 EDT
The workaround of refreshing a file twice doesn't always get the correct cwd.  If doing it under a filter directly it's random.  Sometimes first time works then doing it again messes it up.  Workaround should just be refresh till you can see in the console you have the right cwd.
Comment 17 Kevin Doyle CLA 2007-07-01 23:39:07 EDT
Changing severity to critical based on Michael's comments on bug 194999.

"oh no! I tried to rename a file to index.html. It was overwriting (without
question) the index.html in my root directory, that's worse than deleting the
file...."

Martin can we put out a fix for this right away when one is available?  I've stopped using ftp now because of this issue.
Comment 18 Javier Montalvo Orús CLA 2007-07-02 04:30:42 EDT
Created attachment 72858 [details]
patch

I agree this behaviour is critical. I'll aplly the patch setting as CWD the parent folder of the file/folder to be renamed.
Comment 19 Javier Montalvo Orús CLA 2007-07-02 04:36:17 EDT
The patch is applied now. Could you synchronize org.eclipse.rse.services.files.ftp and verify that it works fine ?
Comment 20 Javier Montalvo Orús CLA 2007-07-02 04:40:51 EDT
*** Bug 194999 has been marked as a duplicate of this bug. ***
Comment 21 Kevin Doyle CLA 2007-07-02 13:39:11 EDT
Looks good after syncing up.
Comment 22 Martin Oberhuber CLA 2007-07-03 09:48:29 EDT
A patch for this has been released to our first maintenance build:
http://download.eclipse.org/dsdp/tm/downloads/drops/M20070702-1115/index.php
as well as the interim update site:
http://download.eclipse.org/dsdp/tm/updates/milestones/

When testing goes well, the patch will also be released to the main update
site. Before this is done, "Check for Updates" will not yet pick up the patch.
Comment 23 Martin Oberhuber CLA 2007-07-03 09:53:03 EDT
Created attachment 72950 [details]
Patched Jar

Attaching a binary patch - drop this into the plugins/ directory of your installation to fix the issue.
Comment 24 Kevin Doyle CLA 2007-07-05 13:21:51 EDT
Verified with I20070705-0600.
Comment 26 Kevin Doyle CLA 2007-07-10 15:53:37 EDT
*** Bug 194207 has been marked as a duplicate of this bug. ***
Comment 27 Martin Oberhuber CLA 2012-05-23 17:17:05 EDT
Comment on attachment 72858 [details]
patch

iplog- since Javier was committer since Aug 2006 already:
http://dev.eclipse.org/mhonarc/lists/dsdp-tm-dev/msg00375.html