Bug 213638 - [jar exporter] create ANT build file for current settings
Summary: [jar exporter] create ANT build file for current settings
Status: RESOLVED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.4   Edit
Hardware: PC Windows XP
: P3 enhancement (vote)
Target Milestone: 3.4 M6   Edit
Assignee: Benno Baumgartner CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed
Depends on:
Blocks:
 
Reported: 2007-12-20 15:37 EST by Ferenc Hechler CLA
Modified: 2008-02-22 06:13 EST (History)
4 users (show)

See Also:


Attachments
add export ant button to Runnable JAR File Export Wizard (11.34 KB, patch)
2007-12-20 15:37 EST, Ferenc Hechler CLA
no flags Details | Diff
runnable jar extension "save as ANT script" (29.05 KB, patch)
2008-01-15 12:05 EST, Ferenc Hechler CLA
no flags Details | Diff
runnable jar extension "save as ANT script" (30.53 KB, patch)
2008-01-17 10:16 EST, Ferenc Hechler CLA
no flags Details | Diff
runnable jar extension "save as ANT script" (56.63 KB, patch)
2008-01-24 17:10 EST, Ferenc Hechler CLA
no flags Details | Diff
runnable jar extension "save as ANT script" (54.46 KB, patch)
2008-01-24 17:35 EST, Ferenc Hechler CLA
no flags Details | Diff
runnable jar extension "save as ANT script" (56.04 KB, patch)
2008-01-25 18:54 EST, Ferenc Hechler CLA
no flags Details | Diff
runnable jar extension "save as ANT script" (56.26 KB, patch)
2008-01-28 14:20 EST, Ferenc Hechler CLA
no flags Details | Diff
fix (53.31 KB, patch)
2008-01-29 09:50 EST, Benno Baumgartner CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ferenc Hechler CLA 2007-12-20 15:37:44 EST
Created attachment 85683 [details]
add export ant button to Runnable JAR File Export Wizard

Build ID: I20071127-0800

Steps To Reproduce:
1. File > Export > Java/Runnable JAR File 
2. Wizard Page "Runnable JAR File Specification" shows up
3. Enhancement: allow exporting the current settings as ANT build.xml file


More information:
ANT 1.7 supports all features we need to create runnable jars (zipfilesets inside a jar).
Eclipse has integrated ANT 1.7 now, so we can use it to create automated build scripts.

I have created a Patch which adds an "Export ANT..." Button to the Wizard. After selecting the save-file-name an ANT script based on the selected launch configuration is created.

The Integration into the GUI is not final. Any Ideas how to better integrate the ANT Export is welcome.
Comment 1 Martin Aeschlimann CLA 2007-12-21 04:31:37 EST
that would be good feature to have (also for the normal JAT exporter)
Comment 2 Benno Baumgartner CLA 2008-01-07 05:28:45 EST
First feedback is:
- fields must start with an 'f' prefix
- Use a DocumentBuilder, see JarPackageWriter for an example
- Let the AntExporter take a JarPackageData, that way we can use it for both export wizards
Comment 3 Dani Megert CLA 2008-01-07 08:24:30 EST
>- Let the AntExporter take a JarPackageData, that way we can use it for both
>export wizards
See bug 117762.
Comment 4 Ferenc Hechler CLA 2008-01-07 15:55:09 EST
Thanks for the feedback.
The quick patch was thought as a basis for discussion, 
so sorry for the poor quality.   :)

About the comments:

> fields must start with an 'f' prefix
will be changed

> Use a DocumentBuilder, see JarPackageWriter for an example
will be used

> Let the AntExporter take a JarPackageData, that way we can use it for both
This might be a bit more difficult. 
The AntExporter gets all of its data from the selected launch-configuration.
This is the Main-Class and the Class-Path.
Both are not part of JarPackageData.

The low-level AntExporter uses the SourceInfo[] structure to represent the classpath. This structure can be filled from the launch-config as well as from the JarPackageData, so the underlying AntExporter can be used for both.
But there must be some preprocessing to derive the low-level info (libs/folders).


I would be interested in some comments about the GUI. 
Is it ok to start the exporter from inside the wizard?
Perhaps it should be done similair to the jardesc export, where the Script is genereted in performFinish() if a filename for the export was set?



Comment 5 Dani Megert CLA 2008-01-08 03:12:54 EST
>I would be interested in some comments about the GUI. 
>Is it ok to start the exporter from inside the wizard?
Not directly i.e. not via button 'Export/Generate Ant'.

>Perhaps it should be done similair to the jardesc export, where the Script is
>genereted in performFinish() if a filename for the export was set?
Exactly. Like in the JAR File or the Plug-in exporter there should be an option:
[ ] Save as Ant script: <drop down list with recent choices> <Browse button>
Comment 6 Benno Baumgartner CLA 2008-01-08 05:31:33 EST
(In reply to comment #4)
> > Let the AntExporter take a JarPackageData, that way we can use it for both
> This might be a bit more difficult. 
> The AntExporter gets all of its data from the selected launch-configuration.
> This is the Main-Class and the Class-Path.
> Both are not part of JarPackageData.

The main class is part of the package data: getManifestMainClass
JarPackageData#getElements() returns IPackageFragmentRoot[] if create with the FatJarWizard, which is the class-path.

Of course, to add this to the normal jar export wizard we need to support all the options from the wizard also in the ant script. But maybe you are interested in doing this as well? And if not then least this is a start.
Comment 7 Ferenc Hechler CLA 2008-01-08 16:32:56 EST
> [ ] Save as Ant script: <drop down list with recent choices> <Browse button>
This sounds good. I will add it.

> JarPackageData#getElements() returns IPackageFragmentRoot[] if create with the
> FatJarWizard, which is the class-path.
I tried this approach first and got stuck very soon, because the IPackageFragmentRoot[] points to the sources and there is a lot of cases to be taken into care to find the interesting compilations.
The simple solution using the launch-configuration classpath has the advantage to behave identical to launching the application from inside eclipse. 
So I think it is a totally different aproach to the normal jar exporter.

> Of course, to add this to the normal jar export wizard we need to support all
> the options from the wizard also in the ant script. But maybe you are
> interested in doing this as well? And if not then least this is a start.
Yes, I am interested in doing this. But one step after another.  :)
The runnable-jar export seems easy to realize so I will first finish this.
The next step might be to support only a subset of the features for the normal jar exporter.
The final version should then also use the JAVAC-Task to compile the sources.


Comment 8 Dani Megert CLA 2008-01-09 03:02:50 EST
>The final version should then also use the JAVAC-Task to compile the sources.
Not for the Jar file exporter as this would lead to a different result than using creating the JAR via wizard or .jardesc.
Comment 9 Benno Baumgartner CLA 2008-01-09 04:16:08 EST
(In reply to comment #7)
> > JarPackageData#getElements() returns IPackageFragmentRoot[] if create with the
> > FatJarWizard, which is the class-path.
> I tried this approach first and got stuck very soon, because the
> IPackageFragmentRoot[] points to the sources and there is a lot of cases to be
> taken into care to find the interesting compilations.
> The simple solution using the launch-configuration classpath has the advantage
> to behave identical to launching the application from inside eclipse. 
> So I think it is a totally different aproach to the normal jar exporter.

Ok, this sounds like you need to implement an IJarBuilder, this would also be another usecase for this new API.
Comment 10 Ferenc Hechler CLA 2008-01-15 12:05:30 EST
Created attachment 86954 [details]
runnable jar extension "save as ANT script"

changes:

 o "Save as ANT script" checkbox instead of an "Export ANT..." button
 o DocumentBuilder is used to create xml
 o history for last saved ANT scripts in combo
 o extended fJarPackage with two fields "fAntScriptSave" and "fAntScriptLocation"

This version works good for me.
It only supports the runnable jar export wizard.
Some code pretty printing must be done.
I will do this after a review of the behaviour.

feri
Comment 11 Benno Baumgartner CLA 2008-01-17 06:34:01 EST
(In reply to comment #10)
> Created an attachment (id=86954) [details]
> runnable jar extension "save as ANT script"

This patch does not compile, looks like you forgot to include the JarPackageData.
Comment 12 Ferenc Hechler CLA 2008-01-17 10:16:55 EST
Created attachment 87161 [details]
runnable jar extension "save as ANT script"

same patch as before, but included missing package
Comment 13 Benno Baumgartner CLA 2008-01-17 12:47:47 EST
- Maybe the UI should read:

[ ] Save as ANT script
    ANT script location: [ << Drop Down >> ] [ Browse... ]

- The wizard should say why the Finish button is not enabled, i.e. 'ANT script location is not valid', or so

- Code needs clean up, as you said

- Please add some unit tests

- If you manage to do all this on less then 50 lines of code then we need no legal review;-)

Comment 14 Ferenc Hechler CLA 2008-01-17 16:11:02 EST
GUI:
will be changed

AntLocation:
 o  check for directory and unwriteable file are already done.
 o  check for empty location will be added.
 o  should there be a check for the extension ".xml"? Perhaps as a warning.

Code cleanup: 
I will give my best...

Unit tests:
I could add some tests for the AntExport class.
I do not know how to test the GUI.
Perhpas the exportAntScript method could be extracted from the Wizard-Page and also be tested.

50 lines of code...
Do you mean 50 lines of additional code?
The patch already has 800 lines.



Comment 15 Benno Baumgartner CLA 2008-01-18 04:06:24 EST
(In reply to comment #14)
> AntLocation:
>  o  check for directory and unwriteable file are already done.
>  o  check for empty location will be added.
>  o  should there be a check for the extension ".xml"? Perhaps as a warning.

No, the dialog shows only xml files, thats good. If the file has no extension then add '.xml', otherwise just ignore the extension. See AbstractJarDestinationWizardPage

> Unit tests:
> I could add some tests for the AntExport class.
> I do not know how to test the GUI.
> Perhpas the exportAntScript method could be extracted from the Wizard-Page and
> also be tested.

We do not test the GUI in this cases. The logic should be separated from the GUI. This will allow to test the logic, and to use the functionality independently of a GUI.

> 50 lines of code...
> Do you mean 50 lines of additional code?
> The patch already has 800 lines.

I was kidding...


Comment 16 Benno Baumgartner CLA 2008-01-24 11:38:03 EST
(In reply to comment #15)
> > 50 lines of code...
> > Do you mean 50 lines of additional code?
> > The patch already has 800 lines.
> 
> I was kidding...
> 

I'm sorry but I have to inform you, that the deadline for submitting IP-reviews is January 31. That means either we'll bring your patch into committable state till end of Tuesday next week or we will have to wait for eclipse 3.5.

Comment 17 Ferenc Hechler CLA 2008-01-24 17:10:04 EST
Created attachment 87815 [details]
runnable jar extension "save as ANT script"

refactorings have been done,
one testcase has been added.
Comment 18 Ferenc Hechler CLA 2008-01-24 17:35:28 EST
Created attachment 87820 [details]
runnable jar extension "save as ANT script"

Some tests have been deactivated in the last patch.
This patch activates all tests.
Comment 19 Benno Baumgartner CLA 2008-01-25 11:13:40 EST
Wow, that was quick! Thanks!

FatJarPackageWizardPage:
- Get rid of UntypedListener class (use anonymous)
- There is a // TODO: [FHe] checks for ant script. Is this obsolete?
- There is a method comment 'Overrides method from WizardDataTransferPage' but method does not override any method
- Comment should be javadoc
- A lot of NLS keys are broken (lower case vs. upper case!)
- Move exportAntScript to FatJarWizard
- Why do you define protected methods? There is no subclass of this internal class.

AntScriptWriter:
- I don't understand your decision to pass some object through the constructor and some as parameters.  IMHO the whole class is not required and can be replaced by one or several helper methods. (in JarAppAntExporter)

JarAppAntExporter:
- Do not use abrv. in class names
- exportAndScript is called from within the tests. In case of a failure an error dialog opens. This will prevent all other tests from finishing. Which is bad. exportAntScript should either throw an exception or the errors must be added to the status, which you already pass in. The callee must then handle the error.
Comment 20 Ferenc Hechler CLA 2008-01-25 18:54:10 EST
Created attachment 87925 [details]
runnable jar extension "save as ANT script"

Followed bennos comments.
Added more export ant tests.
Comment 21 Benno Baumgartner CLA 2008-01-28 08:42:19 EST
Ferenc, it does not work for me:

Given:
package test;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;
public class E1 {
	public static void main(String[] args) {
		Display display = new Display();
		Shell shell = new Shell(display);
		shell.open();
		while (!shell.isDisposed()) {
			if (!display.readAndDispatch())
				display.sleep();
		}
		display.dispose();
	}
}
1. Add swt as external archive to the build path
2. Run E1#main
3. Export runnable jar, save ant script
4. run the exported jar
Is: works as expected
5. run the ant script (I'm using eclipse to run it)
6. run the generated jar
Is:
java -jar test.jar
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
        at sun.security.util.SignatureFileVerifier.processImpl(Unknown Source)
        at sun.security.util.SignatureFileVerifier.process(Unknown Source)
        at java.util.jar.JarVerifier.processEntry(Unknown Source)
        at java.util.jar.JarVerifier.update(Unknown Source)
        at java.util.jar.JarFile.initializeVerifier(Unknown Source)
        at java.util.jar.JarFile.getInputStream(Unknown Source)
        at sun.misc.URLClassPath$JarLoader$2.getInputStream(Unknown Source)
        at sun.misc.Resource.cachedInputStream(Unknown Source)
        at sun.misc.Resource.getByteBuffer(Unknown Source)
        at java.net.URLClassLoader.defineClass(Unknown Source)
        at java.net.URLClassLoader.access$000(Unknown Source)

?
Comment 22 Benno Baumgartner CLA 2008-01-28 08:43:50 EST
Here is the generated ant script for the test case above:

<?xml version="1.0" encoding="UTF-8"?>
<project default="create_run_jar" name="Create Runnable Jar for Project P02">
<!--this file was created by Eclipse Runnable JAR Export Wizard-->
<!--ANT 1.7 is required                                        -->
<target name="create_run_jar">
<jar destfile="C:/Documents and Settings/beb/Desktop/test.jar" filesetmanifest="mergewithoutmain">
<manifest>
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Main-Class" value="test.E1"/>
</manifest>
<fileset dir="C:/e/workspaces/smallws/P02/bin"/>
<zipfileset src="C:/e/I20080122-1600/eclipse/plugins/org.eclipse.swt.win32.win32.x86_3.4.0.v3425.jar"/>
</jar>
</target>
</project>
Comment 23 Ferenc Hechler CLA 2008-01-28 09:26:12 EST
Does the runnable jar created by the export wizard (without ANT) work?
Comment 24 Ferenc Hechler CLA 2008-01-28 09:33:36 EST
> Does the runnable jar created by the export wizard (without ANT) work?
please ignore, just read your description...  sorry.

I assume it has to do with the removed Signer-Files (*.SF) in the destination jar.
Comment 25 Benno Baumgartner CLA 2008-01-28 10:32:28 EST
(In reply to comment #24)
> I assume it has to do with the removed Signer-Files (*.SF) in the destination
> jar.

Remove the file META-INF/Eclipse.sf file from the ANT generated jar file and it works. Do you understand that? Can you fix it?

Comment 26 Ferenc Hechler CLA 2008-01-28 14:20:20 EST
Created attachment 88043 [details]
runnable jar extension "save as ANT script"

added excludes="META-INF/*.SF" to the zipfilesets, this solves the above problem.
Comment 27 Ferenc Hechler CLA 2008-01-28 19:36:14 EST
The problem with the signature files (created by jarsigner) is, that not only each class is signed, but also the whole manifest. 
Merging the manifests together results in invalid signatures. 

Removing the *.SF files from META-INF removes theese signatures.
As a result some libraries will not work because of missing rights, e.g. jce provider in java 6 will not work because of missing "trusted" signers.

feri
Comment 28 Dani Megert CLA 2008-01-29 02:57:24 EST
Eclipse jars are also signed. I doubt we can ship a solution that simply removes the SF files. Also: I'm not sure which SF files you remove. If they are removed from existing code then this might also be a legal issue.
Comment 29 Benno Baumgartner CLA 2008-01-29 04:46:40 EST
(In reply to comment #28)
> Eclipse jars are also signed. I doubt we can ship a solution that simply
> removes the SF files. Also: I'm not sure which SF files you remove. If they are
> removed from existing code then this might also be a legal issue.

The wizard does repack existing library, it does not include the .sf files in the new jar, as I just found out. I think otherwise a generated jar would never run if you repack a signed library. We either don't allow it for this libraries, or we add a warning. Not allowing this would lead to a major user frustration: we forbid something which IMHO is not illegal. Please be aware that this feature passed a legal review.
Comment 30 Dani Megert CLA 2008-01-29 05:05:58 EST
> Please be aware that this feature passed a legal review.
OK. The wizard already warns about that. The other thing is the security issue which the user needs to be aware of.
Comment 31 Benno Baumgartner CLA 2008-01-29 09:50:15 EST
Created attachment 88133 [details]
fix

- Removed the API, if you add this API you need to support it in the IJarExportRunnable returned by createJarExportRunnable. Otherwise people will set this options and wonder why nothing happens.
- Removed all UI from FatJarAntExporter
- let FatJarAntExporter#run throw an CoreException
- cleaned up code, especially the tests
- Added a warning that signature files will not be copied by the exporter. I will also ask the lawyer if she sees any problems with this. 

Now, Ferenc:
- There is a TODO in FatJarAntExporter?
- Can you confirm that you authored all the changes
- Do you agree with the description: The contribution adds the ability to generate an ANT script to the "Runnable JAR" Export Wizard. Based on the information of a Launch-Configuration an ANT script is generated. An ANT script is an xml based script file which can be interpreted by an ANT interpreter. The ANT interpreter will, when interpreting the generated ANT script file, generate the same JAR file as the Runnable JAR Export Wizard generates. The advantage is that this ANT scripts can be used headless, for example within an automated build process.
Comment 32 Ferenc Hechler CLA 2008-01-29 15:42:33 EST
Thank you very much Benno.

I agree with all of your changes.

> There is a TODO in FatJarAntExporter?
The TODO can be removed. 
It is possible to rename a library (*.jar) into any other extension. Every file on the classpath (regardles of its extension) is interpreted as jar-lib.

> Can you confirm that you authored all the changes
I have written all the code changes by myself.

> Do you agree with the description: The contribution ...
The JAR file generated by ANT is nearly the same, but that is nitpicking.

feri
Comment 33 Benno Baumgartner CLA 2008-01-30 11:43:30 EST
I made a request for IP review. Now we need to wait.

Thanks Ferenc!
Comment 34 Philipe Mulet CLA 2008-02-05 12:48:44 EST
Ferenc - Could you explain what is the motivation behind for getting this feature in ?
Comment 35 Ferenc Hechler CLA 2008-02-05 17:26:43 EST
Hello Philippe,

this feature ist very useful for automated Build-Scripts.
Ant scripts can be included into maven and other build environment which are runnable outside of eclipse.
Also you have the possibility to edit the generated script. For example it might be of interest to exclude a signed library and include it via the Class-Path attribute.
Also properties can be excluded via the exclude attribute, because jarred properties have priority over the same file on the Class-Path, which is not what is normally wanted.

Best regards,

    feri

Comment 36 Benno Baumgartner CLA 2008-02-22 05:35:13 EST
fixed > I20080219-1124

Released patch added some minor UI polish