Bug 191367 - with supertransfer on, Drag & Drop Folder from DStore to DStore doesn't work
Summary: with supertransfer on, Drag & Drop Folder from DStore to DStore doesn't work
Status: CLOSED FIXED
Alias: None
Product: Target Management
Classification: Tools
Component: RSE (show other bugs)
Version: 2.0   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 2.0.1   Edit
Assignee: Xuan Chen CLA
QA Contact: Martin Oberhuber CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-06-06 17:08 EDT by Kevin Doyle CLA
Modified: 2007-09-13 10:07 EDT (History)
2 users (show)

See Also:
kmunir: review+
mober.at+eclipse: review+


Attachments
patch to turn off supertransfer by default (1.09 KB, patch)
2007-06-07 09:51 EDT, David McKnight CLA
no flags Details | Diff
fix for super transfer problem (9.81 KB, patch)
2007-08-09 15:18 EDT, Xuan Chen CLA
no flags Details | Diff
updated fix since one of the source file changed. (9.81 KB, patch)
2007-08-09 17:27 EDT, Xuan Chen CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kevin Doyle CLA 2007-06-06 17:08:59 EDT
If I Drag & Drop a folder from one dstore connection to another dstore connection I get the error:

"The file system reported the copy operation failed. You may not have authority to the target folder, or it may be in use"

If I turn off "Enable Fast folder transfer using data compression" from Remote Systems -> Files preference page and do the same folder drag and drop it works fine.

-----------Enter bugs above this line-----------
TM 2.0RC2 Testing
installation : eclipse-SDK-3.3M7
RSE install  : RSE 2.0 RC2
java.runtime : Sun 1.5.0_11-b03
os.name:     : Windows XP, Service Pack 2
------------------------------------------------
Comment 1 David McKnight CLA 2007-06-06 17:12:50 EDT
I didn't realize "enable fast folder transfer" was enabled by default.
Comment 2 Kevin Doyle CLA 2007-06-07 09:05:42 EDT
I checked RC1 and M7 and it wasn't enabled by default.  It changed with RC2.
Comment 3 David McKnight CLA 2007-06-07 09:51:51 EDT
Created attachment 70503 [details]
patch to turn off supertransfer by default

I'm not sure how this changed since it appears the default has always been on - perhaps our initialization of the preferences exposed this, although I would have expected this to be reproduceable after visiting the preference page.   Anyway, I've attached a patch for changing the default to be off.  This should at least allow the normal case to work.
Comment 4 David McKnight CLA 2007-06-07 10:29:16 EDT
For the short term, I'd like to set the default for super transfer to be off (again).  Later we can investigate the supertransfer problem.  I'd like to get this patch reviewed so I can apply it and then defer the defect for later.
Comment 5 Kushal Munir CLA 2007-06-07 11:25:29 EDT
Change looks good. Approved.

I thought we were going to enable supertransfer by default, but since it wasn't on until RC2 (due to whatever reason) and as a result has not been tested much, I'm ok with disabling it for now and fixing the problems later.

We should reenable it after the fix is applied and tested.
Comment 6 David McKnight CLA 2007-06-07 12:11:45 EDT
I've committed the initial default change (changing the default from true to false).  I'm moving the milestone to 2.0.1 to investigate what's going on with super transfer.
Comment 7 Martin Oberhuber CLA 2007-06-08 02:38:23 EDT
ok
Comment 8 David McKnight CLA 2007-07-11 12:32:55 EDT
Reassigning this to Xuan, since the super transfer stuff is archive-related.
Comment 9 Xuan Chen CLA 2007-07-13 01:37:02 EDT
Did some debugging using copying a folder from dstore linux to dstore windows.

I could see a zip file for the folder was uploaded to windows, and then a C_COPY command is issued to copy the source folder from the zip file to its target folder.

In DStoreFileServer#copy()

public boolean copy(String srcParent, String srcName, String tgtParent, String tgtName, IProgressMonitor monitor) throws SystemMessageException
	{
		DataStore ds = getDataStore();
		String srcRemotePath = srcParent + getSeparator(srcParent) + srcName;
		DataElement srcDE = getElementFor(srcRemotePath);
		
		DataElement tgtDE = getElementFor(tgtParent);
		if (tgtDE.getType().equals(IUniversalDataStoreConstants.UNIVERSAL_FILTER_DESCRIPTOR))
		{
			dsQueryCommand(tgtDE, IUniversalDataStoreConstants.C_QUERY_GET_REMOTE_OBJECT, monitor);
		}
                               .
                               .
                              .
                }
		
The srcRemotePath in my case is D:\\tmp\\aaaadd\\supertransfer6599.zip#virtual#/aaaaaasadfc

But the DataElement srcDE = getElementFor(srcRemotePath) call will return a DataElement as the following attributes:

_attributes	String[8]  (id=7406)	
	[0]	"universal.FilterObject"	  <--------------------------------------
	[1]	"1701204208"	
	[2]	"D:\\tmp\\aaaadd\\supertransfer6599.zip#virtual#/aaaaaasadfc"	
	[3]	"D:\\tmp\\aaaadd\\supertransfer6599.zip#virtual#/aaaaaasadfc"	
	[4]	"D:\\tmp\\aaaadd\\supertransfer6599.zip#virtual#/aaaaaasadfc"	
	[5]	null	
	[6]	null	
	[7]	null	

The reason is that in getElementFor() call, we could not find this folder in the _fileElementMap (since it is a virtual folder in a temp file for the transfer), so we create a default DataElement, which is of type universal.FilterObject

protected DataElement getElementFor(String path)
	{
                .
                .
                .   
                DataElement element = (DataElement)_fileElementMap.get(normalizedPath);
		if (element != null && element.isDeleted())
		{
			_fileElementMap.remove(normalizedPath);
			element = null;
		}          
		if (element == null || element.isDeleted())
		{
			DataElement universaltemp = getMinerElement();
			element = getDataStore().createObject(universaltemp, IUniversalDataStoreConstants.UNIVERSAL_FILTER_DESCRIPTOR, normalizedPath, normalizedPath, "", false); //$NON-NLS-1$
		}
                .
                .
                .

		return element;
	}

Since the source of this C_COPY query is of type UNIVERSAL_FILTER_DESCRIPTOR, it will be treated as normal copy command, using "xcopy" for Windows, and "Cp" for Linux, instead of extracting from a archive file.

In my case, the copy command used is:

xcopy "D:\\tmp\\aaaadd\\supertransfer6599.zip#virtual#\\aaaaaasadfc" "D:\\tmp\\aaaadd\\aaaaaasadfc" /Y /K /Q /H

Of course it will not work.
Comment 10 Xuan Chen CLA 2007-08-09 15:18:28 EDT
Created attachment 75789 [details]
fix for super transfer problem

Fixes made:
. Enable supertranfer as default 
. fix the handleCopy() method in UniversalFileSystemMiner.java to handle the special case where the source is a filter object representing a virtual file/folder.
. query the tmp archive file first to cache its information for it to be deleted properly.

Two JUnit testcases had been added to test the following scenario:
. copy from local to DStore Linux and DStore Windows.
. copy from DStore linux to DStore Windows.

I will check in the JUnit testcases separately.
Comment 11 Xuan Chen CLA 2007-08-09 17:27:50 EDT
Created attachment 75799 [details]
updated fix since one of the source file changed.

updated fix since one of the source file changed
Comment 12 Xuan Chen CLA 2007-08-09 22:01:35 EDT
I've committed the patch.
Comment 13 Martin Oberhuber CLA 2007-08-10 05:16:14 EDT
Thanks. Having this fixed is a great achievement.
And having unit tests for it will hopefully keep this functionality stable!
Comment 14 Kevin Doyle CLA 2007-09-13 10:07:44 EDT
Closing this bug as supertransfer now works, but still has some bugs.  New bugs with [Supertransfer] have been opened for the remaining issue's.