Bug 269201 - [jar exporter] ant file produced by Export runnable jar contains absolute paths instead of relative to workspace
Summary: [jar exporter] ant file produced by Export runnable jar contains absolute pat...
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.5   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.5 M1   Edit
Assignee: Ferenc Hechler CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-03-18 10:54 EDT by Thorbjørn Ravn Andersen CLA
Modified: 2014-08-05 06:14 EDT (History)
11 users (show)

See Also:
manju656: review+


Attachments
build.xml file as generated by Eclipse (1.68 KB, text/xml)
2009-03-19 05:37 EDT, Thorbjørn Ravn Andersen CLA
no flags Details
ant script modified to use relative paths (1.91 KB, text/xml)
2009-03-19 05:39 EDT, Thorbjørn Ravn Andersen CLA
no flags Details
patch for ant exporter to use workspace relative paths (19.02 KB, patch)
2009-04-17 15:26 EDT, Ferenc Hechler CLA
no flags Details | Diff
patch for tests to accept new ant scripts (8.60 KB, patch)
2009-04-20 14:47 EDT, Ferenc Hechler CLA
no flags Details | Diff
Alternative Patch (17.65 KB, patch)
2012-10-19 11:36 EDT, Matthias Reinhardt CLA
no flags Details | Diff
patch for ant exporter to use relative paths (20.14 KB, patch)
2012-11-01 05:05 EDT, Ferenc Hechler CLA
no flags Details | Diff
Ferenc's patch updated with minor tweaks (19.96 KB, patch)
2014-06-22 22:40 EDT, Martin Mathew CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Thorbjørn Ravn Andersen CLA 2009-03-18 10:54:19 EDT
Build ID: I20090313-0100

Steps To Reproduce:
1. Open project with a launch configuration
2. File -> Export -> Java -> Runnable JAR file
3. Fill out fields (we use the new Copy required libraries into a sub-folder) and check Save as ANT script with a filename (e.g. c:\test.xml)

Open test.xml with a editor

See that all file and directory references are absolute including C:\...


More information:
It would be very beneficial if the paths put in the ant file are relative (if possible) from the workspace folder, and that that folder was stored in an ant variable

The same with the target directory. 

This will allow us to put the generated ant script in our source repository, instead of having each developer generating the file themselves each time they start a new workspace.

This is such an important facility to us, that I am willing to help getting this ready for the release version of 3.5, if the Eclipse folks do not already have this in place.

Best regards,

/Thorbjørn
Comment 1 Dani Megert CLA 2009-03-18 11:27:52 EDT
>willing to help!
Sure. If you want you can take that bug and provide a patch.
Comment 2 Thorbjørn Ravn Andersen CLA 2009-03-18 15:10:22 EDT
Can you help by providing a build.properties file that works with OS X for building the source bundle.  I cannot locate an online copy of the one used for building the official release :(
Comment 3 Remy Suen CLA 2009-03-18 15:14:52 EDT
(In reply to comment #2)
> Can you help by providing a build.properties file that works with OS X for
> building the source bundle.  I cannot locate an online copy of the one used for
> building the official release :(

Have you developed Eclipse plug-ins before? Did you check out some of the plug-ins from CVS?
Comment 4 Ferenc Hechler CLA 2009-03-19 03:29:19 EDT
Hello Thorbjørn,

I think the first step would be to make examples for the new build file "Templates". 

Just take the generated build files as they are generated now and rewrite them to the format You think about.

Then we should discuss whether the new format has any drawbacks. 
I don´t like the idea to add a checkbox in the GUI for switching the formats, so the new format should replace the old one.

Finally I can take a look at the generator-classes and how much work it will be to change the code. 
You are welcome to implement the changes.   :)

Best regards,

    feri
Comment 5 Thorbjørn Ravn Andersen CLA 2009-03-19 05:36:03 EDT
(In reply to comment #3)
> (In reply to comment #2)
> > Can you help by providing a build.properties file that works with OS X for
> > building the source bundle.  I cannot locate an online copy of the one used for
> > building the official release :(
> 
> Have you developed Eclipse plug-ins before? Did you check out some of the
> plug-ins from CVS?

I have briefly tried earlier (with fatjar actually) but not much.  I would essentially be a new learner :)

Therefore I am glad that Ferenc has offered to look at this :)

Comment 6 Thorbjørn Ravn Andersen CLA 2009-03-19 05:37:32 EDT
Created attachment 129325 [details]
build.xml file as generated by Eclipse

created a project depending on jars in three other projects and created a runnable jar from it.  This is the unmodified ant script generated in the process.
Comment 7 Thorbjørn Ravn Andersen CLA 2009-03-19 05:39:43 EDT
Created attachment 129326 [details]
ant script modified to use relative paths

the revised ant script using paths containing overrideable properties.
Comment 8 Thorbjørn Ravn Andersen CLA 2009-03-19 05:47:38 EDT
I am envisioning something along these lines:

====
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<project default="create_runnable_jar" name="Create Runnable Jar for Project runnableJar with libraries in sub-folder">
	<!--this file was created by Eclipse Runnable JAR Export Wizard-->
	<!--ANT 1.7 is required                                        -->

	<property name="workspace" location="C:/eclipse/workspace-runnableJar/" />
	<echo message="workspace=${workspace}" />
	<property name="destdir" location="c:/xx" />
	<echo message="destdir=${destdir}" />
	<property name="destname" value="runnableJarOutput" />
	<echo message="destname=${destname}" />
	<property name="destjar" location="${destdir}/${destname}.jar" />
	<echo message="destjar=${destjar}" />
	<property name="subdirname" value="${destname}_lib" />
	<echo message="subdirname=${subdirname}" />
	<property name="subdir" location="${destdir}/${subdirname}" />
	<echo message="subdir=${subdir}" />

	<target name="create_runnable_jar">
		<mkdir dir="${destdir}" />
		<jar destfile="${destjar}">
			<manifest>
				<attribute name="Main-Class" value="foo.Bar" />
				<attribute name="Class-Path" value=". ${subdirname}/jt400.jar ${subdirname}/logback-classic-0.9.14.jar ${subdirname}/logback-core-0.9.14.jar ${subdirname}/slf4j-api-1.5.6.jar" />
			</manifest>
			<fileset dir="${workspace}/runnableJar/bin" />
			<fileset dir="${workspace}/ibm-jtopen-5.0/bin" />
			<fileset dir="${workspace}/logback-0.9.X/bin" />
			<fileset dir="${workspace}/slf4j-api-1.5.X/bin" />
		</jar>
		<delete dir="${subdir}" />
		<mkdir dir="${subdir}" />

		<copy todir="${subdir}">
			<fileset file="${workspace}/ibm-jtopen-5.0/jt400.jar" />
			<fileset file="${workspace}/logback-0.9.X/lib/logback-classic-0.9.14.jar" />
			<fileset file="${workspace}/logback-0.9.X/lib/logback-core-0.9.14.jar" />
			<fileset file="${workspace}/slf4j-api-1.5.X/lib/slf4j-api-1.5.6.jar" />
		</copy>
	</target>
</project>
======

I have done the following:

* All absolute file/dir references has been replaced with a property reference and a relative path.  Properties can be overridden from the commandline, allowing this to be location independent.  Perhaps a default value for workspace could be the path from the specified location of the generated file.  I.e. if placed in the root of project A, which is located right under folder W where W is the workspace location, the workspace property default value would be "..".  That would be great for our way of working.

* standalone attribute in xml directive has been set to true as there is no external dependencies.

* target has been renamed to "create_runnable_jar".  Perhaps "createRunnableJar" would be a good one too?

* Copy operation of jar files, has been changed from a copy per file, to a single copy listing all files.

* The destination dir must also have a mkdir to ensure it exists.


The property names are short as this is a proof of concept.  Longer names like "eclipse.workspace" would probably be a good idea for the production version.

Comment 9 Ferenc Hechler CLA 2009-04-17 15:26:49 EDT
Created attachment 132288 [details]
patch for ant exporter to use workspace relative paths

Added patch which changed the ANT scripts as follows:

* Definition of property "${workspace}" which contains the workspace location.
All paths (where possible) are made relative to this location.

* Added mkdir task before creating the jar to ensure existence of destination folder.

Some simple tests seemed to work.

Best regards,

     feri
Comment 10 Ferenc Hechler CLA 2009-04-17 16:05:12 EDT
JUnit Tests are broken.  :-(
Will provide a patch for the tests soon...
Comment 11 Ferenc Hechler CLA 2009-04-20 14:47:37 EDT
Created attachment 132476 [details]
patch for tests to accept new ant scripts

This patch fixes the unit tests to accept the new ANT files
Comment 12 Dani Megert CLA 2010-04-08 11:10:29 EDT
Sorry but the train has left the station again.
Comment 13 Ferenc Hechler CLA 2010-04-12 16:05:24 EDT
(In reply to comment #12)
> Sorry but the train has left the station again.

Have I to do anything else than provide this patch?
Comment 14 Dani Megert CLA 2010-04-13 02:36:56 EDT
>Have I to do anything else than provide this patch?
Not at this point. We simply forgot about this patch and are out of time now. Sorry about that.
Comment 15 Dani Megert CLA 2010-08-02 10:20:08 EDT
Extracting some paths into variables is always useful but using 'workspace' can be misleading because there is no such variable defined in Eclipse out of the box i.e. if I check out the script from the repository and run it, then it will not use my workspace location. Every user has to know that he first needs to define that variable. One can workaround this by also having a shared launch configuration that runs the script, but I think this not nice.

The right approach is to use relative paths if the Ant file is stored inside a project. That way, the script is indeed sharable via repository without additional work(arounds). The UI should also clearly show that:
ANT script location:
(o) Workspace:   [....] [Browse...]  // use absolute paths (with variables)
( ) File system: [....] [Browse...]  // use relative paths

Besides that I'd like to get rid of the code duplication: most of the code that writes the destination stuff is not shared and hence the patch is bigger than it should be.

Unfortunately the tests don't apply in the patch wizard anymore. Sorry.
Comment 16 Stuart Schmukler CLA 2011-03-22 18:16:03 EDT
I would like to see all path navigation like:
    <pathelement location="../myEAR/EarContent/APP-INF/lib/commons-beanutils-1.8.0.jar"/>

to be isolated into ant property defines like:
    <property name="myEAR.location" value="../myEAR"/>

These lines are from my project as exported from Eclipse 3.5 .  The ant export defines some of these path navugation properties, but does not use them!

Stuart
Comment 17 Dani Megert CLA 2011-03-23 03:31:12 EDT
Ferenc are you still planning to provide a patch for this?
Comment 18 Dani Megert CLA 2011-04-12 06:30:13 EDT
Ping!
Comment 19 Ferenc Hechler CLA 2011-04-12 15:47:06 EDT
I will take a look at the necessary changes.
But first of all I have to get my environment set up.
Comment 20 Dani Megert CLA 2011-04-26 04:45:09 EDT
Time's up for 3.7 enhancements.
Comment 21 Ferenc Hechler CLA 2011-06-07 03:13:23 EDT
There is an asymmetry between the jar-file destination and the ant build-file destination.
If a relative path is given, the jar file is relativ to the workspace root,n while the build file is relative to the eclipse folder.

We should change the ant-file relative path also to the workspace root.

Then we have three paths:
 - workspace
 - ant file destination
 - jar file destination

I would define properties for each path and have the following rule:

* Current working directory "." is the ant file destination.
* All paths beyond the current working directory are defined relative.
* If the current working directory is inside the workspace, then all paths which are also inside the workspace are defined relative to the current working directory ("../xxx").
* if the current working directory is outside of the workspace, then all paths which are inside the workspace are defined relative to the workspace property.
* All paths beyond the jar file destination are defined relative to this property.
* all other paths are defined as absolute paths.

The rules will be processed in this order.
Comment 22 Dani Megert CLA 2011-06-10 06:05:36 EDT
(In reply to comment #21)
Sounds good to me. Thorbjørn, does this fit your needs?

I'd like to see in the wizard (info in status area) which approach is used, similar to the JAR file wizard where we tell the user when he stores the JAR file relative to the workspace. It says:
    "The export destination will be relative to your workspace."
Comment 23 jules CLA 2012-03-11 08:34:08 EDT
See bug 373885
Comment 24 Dani Megert CLA 2012-04-16 10:37:20 EDT
Ferenc, do you still plan to work on this?
Comment 25 Lukas Diduch CLA 2012-10-03 15:55:37 EDT
Is the patch from Ferenc/2009-04-17 complete and workable ? (I do not want to setup a devel environment as an eclipse plugin newb only to find out :)

Or did this feature make it into any plugin ? It would the last step in a chain of configurations (at least to me) for being able to completely export a set of related projects and import them to another workspace without having to set up a bunch of launch configurations and reconfigure all the fat jar ant build files.
Comment 26 Dani Megert CLA 2012-10-04 02:39:48 EDT
(In reply to comment #25)
> Is the patch from Ferenc/2009-04-17 complete and workable ? (I do not want
> to setup a devel environment as an eclipse plugin newb only to find out :)
> 

No, that patch needs work.
Comment 27 Ferenc Hechler CLA 2012-10-04 08:47:31 EDT
Oops again?
I´m sorry, that I did not make the changes it in the last year.
I will take a new try to start checking out the JDT and finish the patch.
Comment 28 Lukas Diduch CLA 2012-10-04 14:09:32 EDT
If there's anything i can contribute, e.g. testing let me know.
Comment 29 Matthias Reinhardt CLA 2012-10-19 11:36:27 EDT
Created attachment 222585 [details]
Alternative Patch

As we needed a fast solution for working with relative paths in our version control and build system, I wrote a simple alternative patch to the one by Fernec.

This patch does not require any variables or configuration, and instead assumes that all external operations take place with the working directory set to the folder of the exported xml file.

We would prefer this bug to be fixed upstream instead of having to reapply the patch on every update, so hope this helps.

Regards,
Matthias Reinhardt
Related Designs
Comment 30 Ferenc Hechler CLA 2012-10-31 17:54:03 EDT
I checked out org.eclipse.jdt.ui from the HEAD and am developing under Eclipse 4.2.1
The Tests are all green, so I am up and running...  :-)

I created a small test-project which includes four jars from different locations.


This is the generated build-xml file:

[------ OLD-BUILD-XML -------]
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project default="create_run_jar" name="Create Runnable Jar for Project MainPrj">
    <!--this file was created by Eclipse Runnable JAR Export Wizard-->
    <!--ANT 1.7 is required                                        -->
    <property name="dir.workspace" value=""/>
    <target name="create_run_jar">
        <jar destfile="p:/temp/mainprj.jar" filesetmanifest="mergewithoutmain">
            <manifest>
                <attribute name="Main-Class" value="de.hechler.MainPrj"/>
                <attribute name="Class-Path" value="."/>
            </manifest>
            <fileset dir="P:/runtime-EclipseApplication/MainPrj/bin"/>
            <fileset dir="P:/runtime-EclipseApplication/LibAPrj/libs"/>
            <zipfileset excludes="META-INF/*.SF" src="P:/runtime-EclipseApplication/LibAPrj/libs/libaprj.jar"/>
            <fileset dir="P:/runtime-EclipseApplication/LibCPrj/libs"/>
            <zipfileset excludes="META-INF/*.SF" src="P:/runtime-EclipseApplication/LibCPrj/libs/libcprj.jar"/>
            <fileset dir="P:/TEMP/_TEMP_"/>
            <zipfileset excludes="META-INF/*.SF" src="P:/TEMP/_TEMP_/libbprj.jar"/>
            <fileset dir="C:/temp/libs"/>
            <zipfileset excludes="META-INF/*.SF" src="C:/temp/libs/libdprj.jar"/>
        </jar>
    </target>
</project>
[----------------------------]


The aim is to generate the following output:

[------ NEW-BUILD-XML -------]
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project default="create_run_jar" name="Create Runnable Jar for Project MainPrj">
    <!--this file was created by Eclipse Runnable JAR Export Wizard-->
    <!--ANT 1.7 is required                                        -->
	
	<!-- define folder properties -->
	<property name="dir.buildfile" value="."/> <!-- "P:/runtime-EclipseApplication/MainPrj" -->
	<property name="dir.workspace" value="${dir.buildfile}/.."/> <!-- "P:/runtime-EclipseApplication" -->
	<property name="dir.jarfile" value="P:/TEMP"/> <!-- "P:/TEMP" -->

    <target name="create_run_jar">
        <jar destfile="${dir.jarfile}/mainprj.jar" filesetmanifest="mergewithoutmain">
            <manifest>
                <attribute name="Main-Class" value="de.hechler.MainPrj"/>
                <attribute name="Class-Path" value="."/>
            </manifest>
            <fileset dir="${dir.buildfile}/bin"/>
            <fileset dir="${dir.workspace}/LibAPrj/libs"/>
            <zipfileset excludes="META-INF/*.SF" src="${dir.workspace}/LibAPrj/libs/libaprj.jar"/>
            <fileset dir="${dir.workspace}/LibCPrj/libs"/>
            <zipfileset excludes="META-INF/*.SF" src="${dir.workspace}/LibCPrj/libs/libcprj.jar"/>
            <fileset dir="${dir.jarfile}/_TEMP_"/>
            <zipfileset excludes="META-INF/*.SF" src="${dir.jarfile}/_TEMP_/libbprj.jar"/>
            <fileset dir="C:/temp/libs"/>
            <zipfileset excludes="META-INF/*.SF" src="C:/temp/libs/libdprj.jar"/>
        </jar>
    </target>
</project>
[----------------------------]
Comment 31 Ferenc Hechler CLA 2012-11-01 05:05:51 EDT
Created attachment 223065 [details]
patch for ant exporter to use relative paths

Added a patch which introduces three ant-properties for the following folders:
 - dir.buildfile (folder where the ant-script is)
 - dir.workspace (folder of eclipse workspace)
 - dir.jarfile (folder where the jar-file is)
Comment 32 Ferenc Hechler CLA 2012-11-01 05:23:09 EDT
I have tested the new patch and all three methods create valid build files with relative paths and the jar files generated from the scripts are all running.

The JUnit-Tests are running unchanged, because the assertAntScript() method in FatJarExportTest does not check for the full paths.

If this is okay, I will leave the tests unchanged.

@Lukas: you might now test the patch if it fits your requirements    :-)
Comment 33 Dani Megert CLA 2014-04-23 06:44:39 EDT
Sorry, this slipped through and I don't want to touch this so close to M7.

Please respond in this bug if you're still interested in this feature.
Comment 34 Stuart Schmukler CLA 2014-04-23 11:39:34 EDT
I am still interested in this bug.
Comment 35 Martin Mathew CLA 2014-06-20 01:57:31 EDT
Thanks Ferenc for the patch.

Please add an explicit CoO sign-off comment:
http://wiki.eclipse.org/Development_Resources/Contributing_via_Git#via_Bugzilla
Comment 36 Martin Mathew CLA 2014-06-22 22:40:44 EDT
Created attachment 244414 [details]
Ferenc's patch updated with minor tweaks

The original changes looked fine. The updated patch contains minor changes as listed below:
1. Update the copyright year of all modified files.
2. In UnpackFatJarAntExporter and UnpackJarAntExporter, we are closing "outputStream" in the finally block, hence no need to handle it again in the catch block.
4. In the code we should use the compact assignment form (a= b, i.e. no space on the left of equals). 

This patch can be released once Ferenc comment on the 'Certificate of origin' as mentioned in comment 35.
Comment 37 Ferenc Hechler CLA 2014-06-23 02:37:59 EDT
Thanks to Manju for reworking the code. 

Here my official Certificate of Origin:

This contribution complies with http://www.eclipse.org/legal/CoO.php

I certify that:

1. I have authored 100% of the contribution.
2. I have the necessary rights to submit this contribution, including any necessary permissions from my employer.
3. I am providing this contribution under the license(s) associated with the Eclipse Foundation project I am contributing to.
4. I understand and agree that Eclipse projects and my contributions are public, and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with the license(s) involved.
Comment 38 Markus Keller CLA 2014-06-23 05:08:25 EDT
(In reply to Ferenc Hechler from comment #37)
Ferenc, you also have to sign the CLA, so that the "CLA" image after your name gets a green check mark: https://wiki.eclipse.org/Development_Resources/Contributing_via_Git#Eclipse_Foundation_Contributor_License_Agreement
Comment 39 Ferenc Hechler CLA 2014-06-23 10:16:51 EDT
I signed the CLA, now I´m green  ;)
Comment 40 Martin Mathew CLA 2014-06-23 18:24:53 EDT
Released the patch to 'marsTemp' with: http://git.eclipse.org/c/jdt/eclipse.jdt.ui.git/commit/?h=marsTemp&id=5cd10e87753c3745df255d1bf7ad8bb050290a1b
Comment 41 Jack Gold CLA 2014-07-09 08:48:05 EDT
Is this patch in place publicly?  If not, when will it be available?
Comment 42 Jack Gold CLA 2014-07-09 08:48:29 EDT
Is this patch in place publicly?  If not, when will it be available?
Comment 43 Paul Benedict CLA 2014-07-09 09:33:34 EDT
The branch where this patch was applied no longer exists. Did it get merged into another branch?
Comment 44 Martin Mathew CLA 2014-07-13 20:41:06 EDT
The patch is merged to 'master' branch. The fix is available in the latest Integration build. 
http://download.eclipse.org/eclipse/downloads/
Comment 45 Noopur Gupta CLA 2014-08-05 06:14:52 EDT
Verified for 4.5 M1 with build I20140804-2000.