Bug 363733 - Provide possibility to enhance RequiredPluginsClasspathContainer to deal with Equinox-AdapterHooks
Summary: Provide possibility to enhance RequiredPluginsClasspathContainer to deal with...
Status: RESOLVED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 3.8   Edit
Hardware: PC All
: P3 enhancement with 1 vote (vote)
Target Milestone: 4.3 M5   Edit
Assignee: Curtis Windatt CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed, noteworthy
Depends on:
Blocks:
 
Reported: 2011-11-14 13:24 EST by Thomas Schindl CLA
Modified: 2013-01-10 09:47 EST (History)
8 users (show)

See Also:


Attachments
Service interface (871 bytes, text/plain)
2011-11-14 13:53 EST, Thomas Schindl CLA
no flags Details
Modified plugin classpath class (17.71 KB, text/plain)
2011-11-14 13:54 EST, Thomas Schindl CLA
no flags Details
Example of a contributor (1.39 KB, text/plain)
2011-11-14 13:54 EST, Thomas Schindl CLA
no flags Details
extension point and impl (9.41 KB, patch)
2012-03-01 17:08 EST, Thomas Schindl CLA
no flags Details | Diff
Implementation Patch (12.85 KB, patch)
2012-12-18 04:45 EST, Thomas Schindl CLA
curtis.windatt.public: iplog+
Details | Diff
Sample extender for JavaFX (4.19 KB, text/plain)
2012-12-18 04:50 EST, Thomas Schindl CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas Schindl CLA 2011-11-14 13:24:18 EST
Equinox-Adapter hooks allow people to locate and load external jar-Libaries by registering a so called ClassLoadingHook. 

If such a hook is installed Equinox allows to plugin custom classloaders to e.g. integrate libraries installed on the target system and not allowed to be redistributed (in my case the JavaFX 2.0 libraries).

The problem is that the same logic has to happen when PDE creates the Classpath so that the compiler finds the classes to compile against.
Comment 1 Thomas Schindl CLA 2011-11-14 13:53:06 EST
Created attachment 206972 [details]
Service interface

I'm unable to create patches using egit so I'll attach the files one by one
Comment 2 Thomas Schindl CLA 2011-11-14 13:54:02 EST
Created attachment 206973 [details]
Modified plugin classpath class
Comment 3 Thomas Schindl CLA 2011-11-14 13:54:23 EST
Created attachment 206974 [details]
Example of a contributor
Comment 4 Thomas Schindl CLA 2011-11-14 13:56:09 EST
I might have to add for the time being I ship my own fragment which patches the classpath container in the way shown above and things are working very very smoothly
Comment 5 Vladislav Iliev CLA 2011-11-21 06:14:35 EST
I have a simmilar problem. 
I would like to get access to the class path entry provider in case of OSGi based projects in order to attach java doc to the classpath entry.

I checked Tom's proposal and it works for me.
It leverages on  the OSGi services to contribute the classpath entries –
so each classpath entry provider must register as a service.
So far so good.

The question is : as the contribution part usually is done via extension
points (at least this is the case with org.eclipse.pde.internal.core.RequiredPluginsClasspathContainer),

what are the chances for the Tom’s proposed solution to get into next
Eclipse release?  
Can I stick to it, or I just need to wait or contribute a new extension
point?
Comment 6 Curtis Windatt CLA 2011-11-21 13:11:01 EST
This addition is being considered for the 3.8 release.  We use OSGi services in PDE to provide access to some frameworks (such as the Target Platform).  We typically use extension points for clients to contribute new items.  So an extension point might be preferred, but it isn't necessary.
Comment 7 Thomas Schindl CLA 2011-11-21 13:13:15 EST
I'll rework it as an extension point - just give me some days
Comment 8 Curtis Windatt CLA 2012-02-28 14:51:13 EST
We are approaching API freeze but there hasn't been any activity on this bug.  My PDE time is limited, but if this solves a significant issue for multiple people we can still pursue it for 3.8.
Comment 9 Vladislav Iliev CLA 2012-02-29 02:27:56 EST
+1 for having it for 3.8
Comment 10 Thomas Schindl CLA 2012-03-01 07:30:48 EST
I'm very much for getting this done - but there's more to it than patching PDEClassPathContainer (at least for RCP-Applications). One also has to fix the PDE-Export which I admit I haven't understood fully yet (and so the change I made to https://github.com/tomsontom/e-fx-clipse/blob/master/at.bestsolution.efxclipse.tooling.pde.fragment/src/org/eclipse/pde/internal/core/exports/BuildUtilities.java) is completely wrong.

Who can we add to this bug Curtis to take a look at what is needed for PDE-Export?

I'll try to rewrite my solution to extension points tonight.
Comment 11 Curtis Windatt CLA 2012-03-01 10:11:45 EST
(In reply to comment #10)
> Who can we add to this bug Curtis to take a look at what is needed for
> PDE-Export?
> 
> I'll try to rewrite my solution to extension points tonight.

There are no PDE export/build gurus working on Eclipse anymore :( 

Can you explain more about what in the export needs to change?
Comment 12 Thomas Schindl CLA 2012-03-01 16:10:41 EST
(In reply to comment #11)
> (In reply to comment #10)
> > Who can we add to this bug Curtis to take a look at what is needed for
> > PDE-Export?
> > 
> > I'll try to rewrite my solution to extension points tonight.
> 
> There are no PDE export/build gurus working on Eclipse anymore :( 
> 
> Can you explain more about what in the export needs to change?

Well naturally the export some needs to understand those extra-jars as well, I tried to follow the pde-export but failed to understand how I could teach it the additional jar and I went for an quick and dirty fix by changing the bootclasspath in BuildUtilities.java which is not the 100% correct solution because this way only one version of an external lib can be used.

So the real fix would be needed when we calculate the build-path for the bundle when exported similar to how we do this for RequiredPluginsClasspathContainer.

I'm just working on the conversion of DS stuff to extension points.
Comment 13 Thomas Schindl CLA 2012-03-01 17:08:13 EST
Created attachment 211916 [details]
extension point and impl

This is the proposed API - if you are OK with it I'll work on JavaDoc and Extension Point docu
Comment 14 Curtis Windatt CLA 2012-03-07 10:35:29 EST
(In reply to comment #13)
> Created attachment 211916 [details]
> extension point and impl
> 
> This is the proposed API - if you are OK with it I'll work on JavaDoc and
> Extension Point docu

I did a review of the code and the change is well contained and reasonable. Obviously the documentation on the extension point needs to be finished, explaining what it can be used for.

However, the export issue may keep us from adding this API.  The export uses PDE Build, which calculates its own state and classpath.  So there may be no way for PDE UI to have this API and apply it to the export.  While the UI fix is reasonable, is it worth adding API that will never work for export?  With no PDE Build committers available, working the API into PDE Build will be quite difficult.
Comment 15 Thomas Schindl CLA 2012-03-07 12:14:09 EST
Would a compromise be that you add the Eclipse-ExtensibleAPI header this way one can use a simple fragment to patch out the 2 classes. I currently have to use equinox hooks to repackage the PDE plugin.
Comment 16 Thomas Watson CLA 2012-03-07 12:25:21 EST
(In reply to comment #15)
> Would a compromise be that you add the Eclipse-ExtensibleAPI header this way
> one can use a simple fragment to patch out the 2 classes. I currently have to
> use equinox hooks to repackage the PDE plugin.

This will not help.  If you need to patch the two classes the the PDE plugin in question needs to open its Bundle-ClassPath header to insert a patch jar that a fragment may provide to override the classes in question.

Bundle-ClassPath: patchFor363733.jar, .

Then you would have to provide a fragment that contained the patchFor363733.jar.  Since that jar is before '.' then it would be searched first to find you patched up classes.
Comment 17 Thomas Schindl CLA 2012-03-07 12:58:40 EST
I'll try to dig into the PDE-Build once more
Comment 18 Thomas Schindl CLA 2012-03-07 13:31:25 EST
Walking through an export I see PDE-Build getting into org.eclipse.pde.internal.core.getClasspathMap which consults the also newly introduced IBundleClasspathResolver.
Comment 19 Curtis Windatt CLA 2012-03-07 14:16:03 EST
(In reply to comment #18)
> Walking through an export I see PDE-Build getting into
> org.eclipse.pde.internal.core.getClasspathMap which consults the also newly
> introduced IBundleClasspathResolver.

Can you paste a stack trace of where the interaction occurs.  Note that PDE Build does not depend on on PDE Core/UI (it's the other way around).

Reviewing ClasspathHelper usage I see that we use it to generate the dev.properties which PDE Build does use to determine the classpath.  Hopefully this means the dev.properties will include entries from the extension and no changes to PDE Build are necessary.
Comment 20 Thomas Schindl CLA 2012-03-07 14:40:51 EST
Thread [Worker-14] (Suspended)	
	ClasspathHelper.getDevPaths(IPluginModelBase, boolean, Map) line: 273	
	ClasspathHelper.getDevEntriesProperties(String, boolean) line: 48	
	PluginExportOperation(FeatureExportOperation).getDevProperties() line: 761	
	PluginExportOperation(FeatureExportOperation).setupGenerator(BuildScriptGenerator, String, String, String[][], String) line: 671	
	PluginExportOperation(FeatureExportOperation).doExport(String, String, String, String[][], IProgressMonitor) line: 257	
	PluginExportOperation(FeatureBasedExportOperation).run(IProgressMonitor) line: 50	
	Worker.run() line: 54	

And yes dev.properties does include entries from IBundleClasspathResolver but the export still fails :-(
Comment 21 Thomas Schindl CLA 2012-03-07 14:47:33 EST
the generated dev.properties looks like this:

#
#Wed Mar 07 20:45:34 CET 2012
pluginClasspathContrib=bin,/Users/tomschindl/Desktop/example.jar
myexporter=bin
pde.extender=bin
@ignoredot@=true
org.eclipse.pde.core=bin

and the extension point impl:

public class ExampleBundleClasspathResolver implements IBundleClasspathResolver {

	
	@Override
	public Map getAdditionalClasspathEntries(IJavaProject javaProject) {
		PluginModelManager m = PDECore.getDefault().getModelManager();
		IPluginModelBase model = m.findModel(javaProject.getProject());
		
		Map<IPath, Collection<IPath>> map = new HashMap<IPath, Collection<IPath>>();
		
		for( BundleSpecification bs : model.getBundleDescription().getRequiredBundles() ) {
			if( "myexporter".equals(bs.getName()) ) {
				map.put(new Path("."), Collections.singletonList((IPath)new Path("/Users/tomschindl/Desktop/example.jar")));
			}
			System.err.println(bs.getName());
		}
		
		
		return map;
	}
Comment 22 Pascal Rapicault CLA 2012-04-11 15:39:26 EDT
On the PDE Build side, did you try using the property called 'extra.' in the build.properties [1]? This property is here to allow ppl to add jars on the classpath at compile time though I don't quite remember if this is honoured by PDE Core / UI.

For example, for a jar that has "." as the bundle classpath, the entry would be something like : extra..=foo.jar

[1]http://help.eclipse.org/indigo/topic/org.eclipse.pde.doc.user/reference/pde_feature_generating_build.htm
Comment 23 Thomas Schindl CLA 2012-04-13 10:09:27 EDT
(In reply to comment #22)
> On the PDE Build side, did you try using the property called 'extra.' in the
> build.properties [1]? This property is here to allow ppl to add jars on the
> classpath at compile time though I don't quite remember if this is honoured by
> PDE Core / UI.
> 
> For example, for a jar that has "." as the bundle classpath, the entry would be
> something like : extra..=foo.jar
> 
> [1]http://help.eclipse.org/indigo/topic/org.eclipse.pde.doc.user/reference/pde_feature_generating_build.htm

The problem is that the path to the Jar is not a stable thing but has to be calculated at e.g. by consulting the registry, a preference setting in your IDE, ... .

The logic the extension point tries to resolve is getting dynamic classpath calculation as provided through Equinox hooks to PDE.

my.to.be.hooked.bundle:
  META-INF/MANIFEST.MF: Exports my.example package
  my/example/dummy.class

my.application.bundle:
  META-INF/MANIFEST.MF: Imports my.example package
  my/application/MyTest: references a class my.example.CoolStuff

c:/Program/MySpecialCompany/extra.jar:
  my/example/CoolStuff

At runtime equinox hooks step in and instruct the bundle-classloader of "my.to.be.hooked.bundle" to add an extra.jar and the runtime is happy.

The same now has to happen when the buildpath is calculated. At the moment PDE resolves the my.to.be.hooked.bundle the extension point steps in and adds similar to the equinox hook at runtime the extra.jar to the build path.

Things I'm not sure are:
* is PDE-Build running in an OSGi-Enve
  * when executed in the IDE: yes
  * when executed standalone: ???
* Where does PDE-Build make up the buildpath for javac so that I can do the 
  same i do in the RequiredPluginsClasspathContainer
Comment 24 Thomas Schindl CLA 2012-12-06 18:13:48 EST
I'm coming back to this issue to find out what are the requirements to get this such an enhancement into PDE. The advised patch would provide support for usage inside the IDE, IMHO pde-build is more and more replaced by maven tycho which can be instructed to add jar to the compile time path.

People who want to use this feature need to understand and warned that they can't use PDE-Build. Curtis is there anything beside the missing PDE-Export stopping us to add this stuff?
Comment 25 Curtis Windatt CLA 2012-12-07 12:40:30 EST
(In reply to comment #24)
> I'm coming back to this issue to find out what are the requirements to get
> this such an enhancement into PDE. The advised patch would provide support
> for usage inside the IDE, IMHO pde-build is more and more replaced by maven
> tycho which can be instructed to add jar to the compile time path.
> 
> People who want to use this feature need to understand and warned that they
> can't use PDE-Build. Curtis is there anything beside the missing PDE-Export
> stopping us to add this stuff?

The only two concerns were the impact on PDE build/export and adding complete documentation on the schema and API.

We can move forward with this without solving the issue for PDE Build.  A PDE Build bug should be left open that the documentation can point to.
Comment 26 Thomas Schindl CLA 2012-12-07 12:41:43 EST
Great - I'll prepare all the stuff and create a new patch on the weekend.
Comment 27 Thomas Schindl CLA 2012-12-18 04:45:09 EST
Created attachment 224848 [details]
Implementation Patch

I've reworked the contribution a bit so that one can not only contribute to resolved bundles but also to the bundle the classpath container is created for. 

I've documented the code hopefully enough ;-)
Comment 28 Thomas Schindl CLA 2012-12-18 04:48:40 EST
I've not yet added a PDE-Build remark because I was uncertain where I should add it (maybe in the description of the interface?)

I wasn't sure either on the since tag because the bug is marked for 4.3 but I'm quite sure it is 3.9, not?
Comment 29 Thomas Schindl CLA 2012-12-18 04:50:16 EST
Created attachment 224849 [details]
Sample extender for JavaFX

This sample extender shows how JavaFX is contributed based on the host bundles EE
Comment 30 Thomas Schindl CLA 2013-01-07 05:49:59 EST
Curtis - Any comments?
Comment 31 Curtis Windatt CLA 2013-01-07 09:55:35 EST
(In reply to comment #30)
> Curtis - Any comments?

I haven't forgotten, just busy finishing up Bug 332772 :)
Comment 32 Thomas Schindl CLA 2013-01-07 10:00:19 EST
no problem - it'd like to get this into M5 if possible so that I can plan for it in Kepler.
Comment 33 Curtis Windatt CLA 2013-01-07 14:57:36 EST
Do you want the copyrights to be marked as 'BestSolution.at', 'Thomas Schindl' or something else?
Comment 34 Thomas Schindl CLA 2013-01-07 15:00:06 EST
Yeah that would be nice.
Comment 35 Curtis Windatt CLA 2013-01-07 15:03:02 EST
Also, is there a reason why you have your own interfaces instead of just using the classpath attributes/contributions/access rules from JDT core?
Comment 36 Thomas Schindl CLA 2013-01-07 15:11:40 EST
The only interface duplication i see is the Rule class which is defined also inside the PDEClassPathContainer.
Comment 37 Thomas Schindl CLA 2013-01-07 15:20:39 EST
Is your take is that I should allow to return IClasspathEntry from the extension point? I thought it might be easier to implement this in PDE-Build if because all needed there is the jar which is fed to the compile-path.
Comment 38 Curtis Windatt CLA 2013-01-07 17:24:00 EST
Fixed in master:
http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=7342c9bb2552646017d0cb93f02690d0869e21af

I made significant changes to the doc and API.  Tom, please review that it meets your needs and the doc makes sense to you.

The contributor now returns a list of IClasspathEntry.  This leads to the API being much cleaner.  It allows more flexibility in how contributors can add to the classpath, but I consider that a feature.  Contributors can also take advantage of any improvements in the JDT API.

I'll also look into writing some tests.
Comment 39 Thomas Schindl CLA 2013-01-07 17:47:21 EST
Looks good!
Comment 40 Curtis Windatt CLA 2013-01-08 13:37:16 EST
http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=8ea8b8848cbd2c9e28d371527a5ece0be55d5ecd

Added a test for the API, fixed up some other issues in the tests.

Thanks Tom, closing as FIXED.
Comment 41 Dani Megert CLA 2013-01-09 03:05:41 EST
(In reply to comment #40)
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=8ea8b8848cbd2c9e28d371527a5ece0be55d5ecd
> 
> Added a test for the API, fixed up some other issues in the tests.
> 
> Thanks Tom, closing as FIXED.

Not good: this causes 306 warnings in the official build:
http://download.eclipse.org/eclipse/downloads/drops4/N20130108-2000/compilelogs/plugins/org.eclipse.pde.ui.tests_3.1.400.N20130108-2000/tests.jar.bin.html

Either
- fix the warnings
- revert to 1.4
- adjust the compiler settings for the official build
Comment 42 Curtis Windatt CLA 2013-01-09 09:58:11 EST
For now I've reverted the tests to 1.4
http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=6960e144882c6cc8a57ea9acfb20f1ea824a1ed1
Comment 43 Dani Megert CLA 2013-01-10 06:57:16 EST
(In reply to comment #42)
> For now I've reverted the tests to 1.4
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=6960e144882c6cc8a57ea9acfb20f1ea824a1ed1

Well, this is even worse as it now causes compile errors. You probably don't have your workspace setup correctly i.e. you don't use the JRE(s) as specified in the manifest file(s). You should see a build path warning in your workspace.

Fixed with http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=fd88f5d37212b62812478464718d14a8a67b486a
Comment 44 Curtis Windatt CLA 2013-01-10 09:47:57 EST
(In reply to comment #43)
> Well, this is even worse as it now causes compile errors. You probably don't
> have your workspace setup correctly i.e. you don't use the JRE(s) as
> specified in the manifest file(s). You should see a build path warning in
> your workspace.
> 
> Fixed with
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=fd88f5d37212b62812478464718d14a8a67b486a

Thanks Dani.

I don't have a 1.4 JRE in my workspace but I do have the EE descriptions installed.  Unfortunately the test bundles do not have API tooling builders set so I don't get errors.