Bug 164223 - ftp: big files get only partly transfered
Summary: ftp: big files get only partly transfered
Status: CLOSED FIXED
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: 1.0   Edit
Hardware: PC Windows XP
: P1 major (vote)
Target Milestone: 1.0   Edit
Assignee: Martin Oberhuber CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords: greatbug
Depends on:
Blocks:
 
Reported: 2006-11-11 13:04 EST by Michael Scharf CLA
Modified: 2006-11-13 06:55 EST (History)
1 user (show)

See Also:

Attachments
patch that fixes the problem (1.75 KB, patch)
2006-11-12 09:53 EST, Michael Scharf CLA 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Scharf CLA 2006-11-11 13:04:38 EST 
I was using 1.0.0.v20061110 with ftp and when I transfer big files, the file is not fully transfered.
A few bytes at the end are missing...

I tried different FTP hosts -- with the same result....

It feels like a close or flush is missing at the end of the transfer.....
Comment 1 Michael Scharf CLA 2006-11-11 13:10:54 EST 
Forgot to mention: transfer is in direction to the ftp site. (the file on the ftp host is corrupt)
Comment 2 Martin Oberhuber CLA 2006-11-11 13:55:47 EST 
How big is the file?
JVM version etc.?
Comment 3 Martin Oberhuber CLA 2006-11-11 13:59:12 EST 
Also, do you see any errors logged?
What does the FTP Console View look like?
Comment 4 Martin Oberhuber CLA 2006-11-11 14:01:36 EST 
Finally, how did you upload? Drag&drop (from where?), Copy&paste or other?
Comment 5 Michael Scharf CLA 2006-11-12 09:48:31 EST 
The problem occurs for binary files transfered between Unix and Windows.
It actually transfers binary files in text mode and therefore converts
line endings. A Transfer from Windows to Unix shortens the file.

I figured out the problem: setFileTransferMode was called instead of setFileType.

In FTPService.upload is the following code:

			if (isBinary)
				ftpClient.setFileTransferMode(FTP.BINARY_FILE_TYPE);			
			else
				ftpClient.setFileTransferMode(FTP.ASCII_FILE_TYPE);

The problem is that, setFileTransferMode takes one of the following constants
    FTP.STREAM_TRANSFER_MODE
    FTP.BLOCK_TRANSFER_MODE
    FTP.COMPRESSED_TRANSFER_MODE

To set the file type use
			if (isBinary)
				ftpClient.setFileType(FTP.BINARY_FILE_TYPE);			
			else
				ftpClient.setFileType(FTP.ASCII_FILE_TYPE);

That actually fixes the problem for me.

This is a good example why enumerations introduces in java 1.5 are a good idea and why int constants are really really bad
It's so easy to get it wrong and the compiler does not complain if you use the wrong constants....

BTW. in case of doubt the file transfer should always be binary. I would make it a property
of the FTP client or preference entry if line endings should be converted or not.....

setFileType (like setFileTransferMode) returns a false if the call failed. Maybe this should be checked too.....
Comment 6 Michael Scharf CLA 2006-11-12 09:53:33 EST 
Created attachment 53714 [details]
patch that fixes the problem

Note: there are 4 wrong calls to setFileTransferMode
Comment 7 Michael Scharf CLA 2006-11-12 09:54:32 EST 
This is a showstopper I think!
Comment 8 Martin Oberhuber CLA 2006-11-13 04:52:00 EST 
Applied the patch, and respun RSE 1.0. The fix can be downloaded from
http://download.eclipse.org/dsdp/tm/downloads/drops/R-1.0-200611121600/
for verification.

This is a great bug Michael -- you not only discovered the issue and documented it, but also fixed it and provided a patch over the weekend. 
Thanks!
Comment 9 Martin Oberhuber CLA 2006-11-13 06:55:42 EST 
Verified with 1.0 against FileZilla, Closing.