Bug 189352 - [dstore] Copy & Paste transfers binary although text mode specified
Summary: [dstore] Copy & Paste transfers binary although text mode specified
Status: RESOLVED FIXED
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: 2.0   Edit
Hardware: PC Windows XP
: P2 major (vote)
Target Milestone: 2.0   Edit
Assignee: Kushal Munir CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 269169
  Show dependency tree
 
Reported: 2007-05-28 01:06 EDT by Martin Oberhuber CLA
Modified: 2012-05-23 17:42 EDT (History)
3 users (show)

See Also:
ddykstal.eclipse: review+


Attachments
Attachment with bug fix (4.25 KB, patch)
2007-06-04 18:21 EDT, Kushal Munir CLA
no flags Details | Diff
Check for Linux/Unix system type and replace line end character (5.09 KB, patch)
2007-06-04 18:37 EDT, Kushal Munir CLA
no flags Details | Diff
Replace line end character if source and target line end characters do not match (5.76 KB, patch)
2007-06-04 23:42 EDT, Kushal Munir CLA
mober.at+eclipse: iplog-
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Martin Oberhuber CLA 2007-05-28 01:06:40 EDT
Preferences > Remote Systems > Fils, specify *.sh files to be binary.
Editing a *.sh file on Windows with CRLF endings.
In Package Explorer, select *.sh file and "Copy".
Navigate to remote dstore-linux (or ssh) folder, press paste.
Open a shell on destination folder, type
  od -c | head -5
See that there are still CRLF endings.

I had supposed that Text mode transfer should fix the line endings?
Or what is "Text mode transfer" supposed to apply to?
This is important, since CRLF *.sh files don't execute properly on Linux.

-----------Enter bugs above this line-----------
TM 2.0RC1 Testing
installation : eclipse-platform-3.3M6 (I20070323-1616), cdt-4.0M5, emf-2.3M5
RSE install  : download RSE 2.0RC1 runtime-all + terminal
java.runtime : Sun 1.6.0-b105
os.name:     : Windows XP 5.1, Service Pack 2
------------------------------------------------
systemtype   : Linux-ssh (dstore-processes)
targetos     : SUSE Linux Enterprise Server 10 (ppc)
targetuname  : Linux build 2.6.16.27-0.9-ppc64 #1 SMP ppc64 GNU/Linux
targetvm     : ibm-java2-sdk-5.0-0-4.0-linux-ppc
------------------------------------------------
Comment 1 David McKnight CLA 2007-05-28 09:53:15 EDT
Kushal, would any of the changes you made have caused this?
Comment 2 Rupen Mardirossian CLA 2007-05-28 17:00:42 EDT
I've done some debugging and found out that the preferences are being honored.  By setting BP at the method below  

public boolean download(String remoteParent, String remoteFile, File localFile, boolean isBinary, String hostEncoding, IProgressMonitor monitor) throws SystemMessageException

in both dstore-linux and FTP (assuming SSH would be the same)

If the file transfer mode is set to binary the isBinary variable is true.
If the file transfer mode is set to text the isBinary variable is false.

-----------Enter bugs above this line-----------
TM 2.0M7 Testing
installation : eclipse-SDK-3.3M7 
RSE install  : RSE 2.0RC1
java.runtime : Sun 1.5.0_06-b05
os.name:     : Windows XP 5.1, Service Pack 2
------------------------------------------------
Comment 3 Kushal Munir CLA 2007-06-04 17:05:51 EDT
This never actually worked. It was not as a result of the changes I had made.

The problem in dstore is that we read in a set of bytes, which include the Windows (or whatever the local platform happens to be) line end characters, convert to the remote file encoding and then do the transfer. This preserves the line end characters of the local platform during the transfer, which leads to the problem.  
Comment 4 Kushal Munir CLA 2007-06-04 18:21:47 EDT
Created attachment 70046 [details]
Attachment with bug fix

Bug fix that strips out the line end characters and replaces with line end character of target (if it is Unix/Linux).
Comment 5 Kushal Munir CLA 2007-06-04 18:37:58 EDT
Created attachment 70049 [details]
Check for Linux/Unix system type and replace line end character
Comment 6 Kushal Munir CLA 2007-06-04 18:38:51 EDT
Comment on attachment 70049 [details]
Check for Linux/Unix system type and replace line end character

Added description in comments at the top of the files to describe the changes
Comment 7 David Dykstal CLA 2007-06-04 21:25:15 EDT
Kushal -- I'm not sure about this patch. If I'm reading it correctly it replaces CRLF with LF regardless of the source system line end convention. While this probably going to be right most of the time, I'm not sure it is right all of the time.
Comment 8 Kushal Munir CLA 2007-06-04 23:42:45 EDT
Created attachment 70071 [details]
Replace line end character if source and target line end characters do not match

Checks the local line end character and determines the remote line end character from the given remote system type. This will replace line end characters so long as they do not match (i.e. from Windows to Linux/Unix or vice versa).
Comment 9 Kushal Munir CLA 2007-06-04 23:44:54 EDT
Dave, please see the modified changes. Line end characters will now be replaced if they don't match. So if a file is transferred from Windows to Linux/Unix or vice versa, line end characters will be replaced with that of the destination line end characters. For transfer between same system types, no replacement will be done (e.g. Windows to Windows).
Comment 10 David Dykstal CLA 2007-06-05 09:16:32 EDT
Ahh! much better.
Comment 11 Kushal Munir CLA 2007-06-05 10:07:05 EDT
Patch applied.
Comment 12 Martin Oberhuber CLA 2008-08-13 13:20:06 EDT
[target cleanup] 2.0 RC2 was the original target milestone for this bug
Comment 13 Testo Nakada CLA 2009-03-18 06:42:47 EDT
I tested using SFTP, it still transfer .sh file as binary while FTP works as expected
Comment 14 Martin Oberhuber CLA 2009-03-18 07:37:34 EDT
You are right, the original fix was for dstore only.
I have created bug 269169 to fix it for SSH as well.
Thanks for the report!
Comment 15 Martin Oberhuber CLA 2012-05-23 17:42:30 EDT
Comment on attachment 70071 [details]
Replace line end character if source and target line end characters do not match

Kushal became committer on 20-Apr-2006 so the contribution was done as committer:
http://dev.eclipse.org/mhonarc/lists/dsdp-tm-dev/msg00169.html