Bug 20599 - [ExternalTools] F3 behaves differently than F2 with variables in "Tool Arguments"
Summary: [ExternalTools] F3 behaves differently than F2 with variables in "Tool Argume...
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Ant (show other bugs)
Version: 2.0   Edit
Hardware: PC other
: P2 major (vote)
Target Milestone: 2.0 F4   Edit
Assignee: Ryan Cooper CLA
QA Contact:
URL:
Whiteboard:
Keywords: readme, ui
Depends on:
Blocks:
 
Reported: 2002-06-18 17:00 EDT by Michael R. Head CLA
Modified: 2002-09-18 13:59 EDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael R. Head CLA 2002-06-18 17:00:51 EDT
My current build process requires me to call ant with various directory paths
set as "-D" parameters, which worked in F2. In F3, this is broken because of how
variables are being treated. Quotes are being passed in around the expanded
variable.

Example (using F3):

Buildfile: build.xml

all:
     [echo] test.variable="/home/burner/.eclipse/testcase"

BUILD SUCCESSFUL
Total time: 1 second

And here's what I expect (and what F2 did for me):

Buildfile: build.xml

all:
     [echo] test.variable=/home/burner/.eclipse/testcase

BUILD SUCCESSFUL
Total time: 2 seconds


Here is how my external tool is setup (as copied from the external tools
configuration window):

Location: /usr/bin/ant
Arguments: -Dtest.variable=${project_loc:/testcase}
Directory: ${project_loc}

And here is the example build.xml file I used to get this result:

<?xml version="1.0"?>

<project name="test" default="all">
   <target name="all">
   	<echo message="test.variable=${test.variable}"/>
   </target>
</project>


I don't know a decent way around this, besides not using the variables and hand
coding the paths to my projects (which I am rather loathe to do in the long run).

mike
Comment 1 Rodrigo Peretti CLA 2002-06-18 17:20:18 EDT
Tried with build 20020618 and I cannot reproduce using Windows. Will try on 
Linux. Here is what I get on Windows XP Pro:

all:
     [echo] test.variable=C:\eclipse\test\antworkspace\Test
Comment 2 Rodrigo Peretti CLA 2002-06-18 18:13:32 EDT
Could reproduce with F3 and 20020618 under Linux. I do not believe the problem
is in org.eclipse.ant.core because the external tool in this case is not an Ant
script but the Ant launcher that ships with a binary distribution of Ant. Needs
more investigation to see what has changed between F2 and F3 in this area.
Comment 3 Rodrigo Peretti CLA 2002-06-18 18:35:54 EDT
I've just installed F2 (20020602) and it also gives the location using quotes -
so, I see no differences between F2 and F3. Could you try again and make sure F2
gives you the location without the quotes? Thanks.
Comment 4 Michael R. Head CLA 2002-06-18 20:20:35 EDT
I have just downloaded F2 (20020602) again, and unzipped it to a new directory
and created a new workspace.

I created a new simple project (with the wizard) called "testcase", in the
project I created a build.xml file with the code copy/pasted from this page. 

I then created a new "external tool" entry with the following details:
Location: /usr/bin/ant
Arguments: -Dtest.variable=${project_loc:/testcase}
Directory: ${project_loc}

I got this output:
Buildfile: build.xml

all:
     [echo] test.variable=/home/burner/eclipse/F2/eclipse/workspace/testcase

BUILD SUCCESSFUL
Total time: 2 seconds

I'll shortly post my results with a newly download and unzipped F3...
Comment 5 Michael R. Head CLA 2002-06-18 20:24:24 EDT
I just followed the same steps I just described with F3 (20020612), and get this
result:

Buildfile: build.xml

all:
     [echo] test.variable="/home/burner/eclipse/F3/eclipse/workspace/testcase"

BUILD SUCCESSFUL
Total time: 2 seconds

Comment 6 Michael R. Head CLA 2002-06-18 20:29:31 EDT
I suppose I should provide some more details of the system I'm running on:

Debian sid/unstable (I can provide library version details if necessary)

Sun's JDK1.4.0
java -version:
java version "1.4.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)

Ant version 1.5beta1
/usr/bin/ant -version:
Apache Ant version 1.5Beta1 compiled on May 26 2002

mike
Comment 7 Rodrigo Peretti CLA 2002-06-19 14:29:30 EDT
It seems that the UI has done some work in this area (adding quotes to the 
working dir). Moving to Platform UI for comments.
Comment 8 Ryan Cooper CLA 2002-06-19 15:31:59 EDT
The quotes were added to handle paths that include spaces. Unfortunately we did 
not test on Linux. Investigating further...
Comment 9 Michael R. Head CLA 2002-06-19 15:57:27 EDT
Here's a shell script that also exemplifies the problem I just cooked up to
focus on what F3 is doing differently (and pull out any possible ant :
#!/bin/sh

echo _=$_ 
echo 0=$0 
echo 1=$1
[ -f "$1" ] && echo File exists || echo File does not exist

I created test.sh in my testcase project, and create a new External Tool with
the following properties:

Location: ${workspace_loc:/testcase/test.sh}
Arguments: ${resource_loc}
Directory: ${workspace_loc:/testcase}

I run this by clicking on the test.sh file in the testcase project and running
the "test" external tool.

And when I run this in F3, it says:
_=/bin/sh
0=/home/burner/.eclipse/testcase/test.sh
1="/home/burner/.eclipse/testcase/test.sh"
File does not exist


In F2, it says:
_=/bin/sh
0=/home/burner/.eclipse/testcase/test.sh
1=/home/burner/.eclipse/testcase/test.sh
File exists

mike
Comment 10 Ryan Cooper CLA 2002-06-19 16:28:29 EDT
There are three potential fix options:

1) Check to see if a path has spaces, and only enclose in quotes if it does. 
This will eliminate regression, and users will still be able to use paths with 
spaces in Windows, but will not be able to successfully use paths with spaces 
in Linux. This will solve Micheal's problem with minimal effort.

2) Put platform-specific code in org.eclipse.ui.externaltools. Use quotes to 
enclose paths in Windows, and use the proper method to handle paths with spaces 
for other OSs. This would solve the problem for users of both platforms, but in 
my opinion is a little more risky and would require lots of testing for 
different platforms. This could overlap with option 1, but that would be a lot 
of work.

3) Ask core to provide API to pass AntRunner the arguments as an array of 
Strings, instead of as a single String of arguments seperated by spaces. Thsi 
would be the most complete solution, but would require non-trivial changes to 
our code. Recommended as post-2.0 solution.
Comment 11 Rodrigo Peretti CLA 2002-06-19 16:34:22 EDT
Ryan, about #3 - in the current test case AntRunner is not being used. The 
external tool is /usr/bin/ant and not the script itself.
Comment 12 Nick Edgar CLA 2002-06-19 21:02:49 EDT
Marked for F4 since this is a regression.

Option 1 seems the safest.

Also note that it's better to use the Runtime.exec(String[], ...) variants 
rather than exec(String, ...).  The reason is that the String variants parse 
the string into arguments using StringTokenizer, which does not understand 
quotes.  It merely tokenizes based on white space.
So, Runtime.exec("someProg \"the parameter\"") ends up being equivalent to 
Runtime.exec(new String[] {"someProg", "\"the", "parameters\""}).
Now, depending on how the natives are written, they may get glued back 
together properly, but they might not.

Comment 13 Nick Edgar CLA 2002-06-19 21:48:09 EDT
Need a proposed fix before seeking approval.
Comment 14 Ryan Cooper CLA 2002-06-20 09:39:48 EDT
Suggested fix:

Change DefaultRunnerContext as shown (adding call to hasSpace(String)), and add 
method hasSpace(String) as shown below.

Changing our calls to Runtime.exec(...) to use an array of Strings instead of a 
single string would require non-trivial changes to DefaultRunnerContext [see 
expandVariables(...), expandVariable(...)]. Since regression is avoided with 
the fix below, I'm not convinced that switching to Runtime.exec(String[], ...) 
is a good idea at this point.

/**
 * Helper method to add the given variable string to the given
 * string buffer if the string is not null. Adds enclosing quotation
 * marks if addQuotes is true.
 * 
 * @param var the variable string to be added
 * @param buf the string buffer to which the string will be added
 * @parman addQuotes whether or not to add enclosing quotation marks
 */
private void appendVariable(String var, StringBuffer buf, boolean addQuotes) {
	if (var != null) {
		if (addQuotes && hasSpace(var))
			buf.append("\""); //$NON-NLS-1$
		buf.append(var);
		if (addQuotes && hasSpace(var))
			buf.append("\""); //$NON-NLS-1$			
	}
}
	
/**
 * Returns whether or not the given string contains at least one space.
 * 
 * @return true if the given string contains at least one space, false otherwise
 */
private boolean hasSpace(String var) {
	int index = var.indexOf(' ');
	if (index >= 0)
		return true;
	else
		return false;
}
Comment 15 Ryan Cooper CLA 2002-06-20 11:11:09 EDT
Alternate solution:

Never automatically add quotes. The user can add quotes to the commnad line 
arguments manually if they wish to use quotes.

Change required:
in DefaultRunnerContext.getExpandedArguments(), change call to expandVariables
(String, boolean) to use false instead of true.

if (expandedArguments == null)
	expandedArguments = expandVariables(tool.getArguments(), false);
Comment 16 Simon Arsenault CLA 2002-06-20 11:31:42 EDT
No to the last alternative (i.e letting user add quotes). It's the 
responsibility of the external tools plugin to format the paths in such a way 
that is acceptable for the platform.
Comment 17 Ryan Cooper CLA 2002-06-21 09:14:46 EDT
I got the same results as Rodrigo on WIndows: no quotes

[echo] test.variable=D:\0619i\eclipse\workspace\testcase
Comment 18 Nick Edgar CLA 2002-06-21 10:39:11 EDT
Approved to fix: quotes will only be added when there is a space in the value.
This will work fine on Windows for both variables with spaces and those without.
On Linux, it will work for variables without spaces, but the quotes will still 
be taken literally when added for variables with spaces.

Added bug 20797 to revisit this post 2.0.
Comment 19 Nick Edgar CLA 2002-06-21 10:57:00 EDT
The limitation on Linux needs to be documented in the readme.
Comment 20 Nick Edgar CLA 2002-06-21 11:10:45 EDT
In the readme, should mention that a workaround on Linux for variables with 
spaces is for the script to strip off quotes.
Comment 21 Simon Arsenault CLA 2002-06-21 11:48:43 EDT
Ryan implemented the suggested code changes

Checked by Simon and Tod