Bug 3074 - [osgi] allow external libs onto the platform classpath? (1GFLHIN)
Summary: [osgi] allow external libs onto the platform classpath? (1GFLHIN)
Status: RESOLVED WONTFIX
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Runtime (show other bugs)
Version: 3.0   Edit
Hardware: All All
: P3 enhancement with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: platform-runtime-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 10932 14020 19327 50371 (view as bug list)
Depends on:
Blocks:
 
Reported: 2001-10-10 22:49 EDT by Jeff McAffer CLA
Modified: 2009-08-18 17:39 EDT (History)
13 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff McAffer CLA 2001-10-10 22:49:00 EDT
There are usecases whre the user has some Java library installed on their 
machine (e.g., JDBC)
for which they do not have a plugin.  Creating a plugin is difficult as the 
install location will be
different for all machines.  Since these libs come as part of some other 
install we cannot mandate
that they be in any particular location, form, ...

The standard Jvaa answer is to put these libs on the app classpath and be done 
with it. We drop the 
app classloader when we run the platform.  

Alternatives for users are :
	- create a plugin
	- add the lib to their plugin
	- add the lib to the bootpath

Alternatives for us
	- leave the app class loader as the parent of the bootloader
	- make some mechanism for people to add libs to the platform path
	- allow users to spec "external" libs on their plugin classpath
	- use environment vars on the classpath in a plugin
		e.g, library name="$INSTALL$/lib.jar"
		where $INSTALL$ is an actual OS environment var which id's the 
install locatoin of the 
		external product.  This would be ignored if the lib.jar was not 
found (as per normal class
		path processing)


NOTES:

JM (6/27/2001 6:46:57 PM)
	This requirement surfaced in the news group.  Basically with our 
current structure, if people are not
	willing to supply plugins, eclipse developers cannot use the code.  
There are cases where you can
	create the plugin for them but there are clearly cases where people are 
expecting their own install 
	structure.  



ARCH (7/11/01 10:11:59 AM)
	Investigate but at the moment not required for 1.0
Comment 1 DJ Houghton CLA 2001-10-24 06:43:16 EDT
PRODUCT VERSION:

125
Comment 2 Pat McCarthy CLA 2002-01-31 13:30:11 EST
Question on status of this item.

Is this something we can expect to see resolved in V2.0?  The need for tools to 
be able to use external JARs, especially that used to support DB access, is 
real.  

I found and reviewed the newsgroup discussion from last June regarding bootpath 
and how it might be used to resolve the problem.

We have not seen anything beyond the bootpath comment that tells us that this 
problem will resolved or at least guidance on how a tool can solve this problem 
in a reasonable fashion.

Any news/comments - or is bootpath the message we push forward to ISVs/dev 
teams?

Thanks,

Pat Mc.
Comment 3 David Wegener CLA 2002-05-02 10:58:13 EDT
It may be more appropriate to provide a classpath extension mechanism for 
plugins.  This would allow two different plugins to each use conflicting 3rd 
party jars.  The fact that each plugin gets its own class loader could actually 
help in this situation.  A new preference page (under Workbench?) would provide 
a list of all plugins.  Selecting a plugin would allow for defining extensions 
to the plugin classpath.  The plugin class loader would need to be changed to 
append/prepend the extension classpath onto the plugin classpath derived from 
the plugin.xml file.  This would allow for the configuration of 3rd party jars 
needed for JDBC access, XML parsers, etc.  Optionally the preference page would 
provide a workbench classpath extension that was appended/prepended to the 
classpath for each plugin.
Comment 4 John Arthorne CLA 2002-05-02 14:36:23 EDT
This would be a valuable problem to fix if we can find time.  Many plugins 
wishing to make use of third party tools run into this problem, and it 
frequently comes up on the newsgroup.  The workaround to is to define your own 
classloader or use the bootpath, but it would be nice if we provided a cleaner 
way to do this.

Moving back to the "Inbox".
Comment 5 Bob Foster CLA 2002-05-02 20:09:53 EDT
I had some practical experience with this problem where my plugin needed to use
a different version of Xerces than the version used by several org.eclipse
plugins my plugin depends on. (I say "different" because it shouldn't matter
whether it is a later or earlier version.) I recommend this as a test case for
any proposed solution.
Comment 6 Rodrigo Peretti CLA 2002-06-05 12:06:24 EDT
*** Bug 19327 has been marked as a duplicate of this bug. ***
Comment 7 DJ Houghton CLA 2002-09-27 11:37:41 EDT
*** Bug 10932 has been marked as a duplicate of this bug. ***
Comment 8 Bea Petrovicova CLA 2003-08-20 04:16:26 EDT
I have problem with jar library needed 
durring BOOT of java virtual machine. 
This library needs to be present in ROOT 
classpath, due JAAS modules must be visible
from this classpath at boot time and cannot 
be added durring runtime unlike security 
providers.

Is there any other solution than using ugly 
-Xbootclasspath/a:c:/eclipse/... solution?
Comment 9 John Arthorne CLA 2004-01-26 11:22:53 EST
*** Bug 50371 has been marked as a duplicate of this bug. ***
Comment 10 John Arthorne CLA 2004-01-26 11:24:40 EST
Moving this bug to Equinox component for comment/consideration. Class loading is
now in the hands of the underlying OSGi implementation. Jeff, we've been saving
this bug for you :)
Comment 11 Jeff McAffer CLA 2004-01-26 16:20:43 EST
Very much a friend of bug 30919.  
Comment 12 Jeff McAffer CLA 2004-03-22 16:13:44 EST
Actually I am going to mark this a dup of bug 30919

*** This bug has been marked as a duplicate of 30919 ***
Comment 13 Ben Youngdahl CLA 2004-06-03 11:20:31 EDT
This bug is marked as a duplicate of 30919, and 30919 is marked as fixed, but if
you read both bugs you will see that the problems raised in this original bug
report are *not* fixed with the resolution of 30919.  

Would you please consider reopening this bug, as it is not solved by the
resolution of 30919.
Comment 14 Ben Youngdahl CLA 2004-06-03 11:27:53 EDT
To further elaborate, the resolution in 30919 only allows external JARs to be
referenced by Eclipse plug-ins if they are deposited in the Java Runtime
Environments "ext" folder.  This original bug was looking for the ability to
reference JARs which might be at some arbitrary point in the filesystem.
Comment 15 Jeff McAffer CLA 2004-06-03 14:42:40 EDT
reopening to consider post 3.0.  As you can see by the history of this we are 
not keen on it.

If we allow something like this we end up in a number of problem areas:
- plugins are no longer components as they do not carry their payload.
- related: version management and API declarations are rendered meaningless
- absolute paths in the plugin.xml/manifest.mf are brittle
- next request will be for environment variable (or some such) support in the 
class path entries
Comment 16 Igor Fedorenko CLA 2004-06-03 22:35:38 EDT
To add my two cents. With 3.0 it should be possible to use external jar files by
dynamically creating plugins/bundles (disclaimer: it worked with M5 or M6 but I
did not try with recent builds). I am not saying that I am happy with this
solution as it in my oppinion requires too much effort for such common problem
but this is better than nothing.
Comment 17 Christophe Avare CLA 2004-06-08 11:37:31 EDT
It is sad that this issue will not be resolved soon.
This means that many librairies will not be available for the Eclipse platform.
Mainly, Jeff arguments do not take into accounts librairies whose licensing and
distribution terms restrict the way the code can be distributed.
For example the Java Advanced Imaging, Java3D and the like cannot be delivered
as plugins without breaking the binary license agreement from Sun.
More, many well known Open Source librairies are build and install themselves as
regular extensions.
While, in this case, one can always provide an Eclipse packaging, this will lead
to many sources for the same librairies, the worst case being having to install
the same version of a given library one for feature X, another from Y, and so on.
Until there is a shared repository and a large effort from all the Java actors
to provide Eclipse plugins or bundles we will have to rely on external librairies.

I personally doubt that Sun will either change its licensing terms or distribute
its librairies as Eclipse plugins in a foreseeable future!
Comment 18 Jeff McAffer CLA 2004-06-08 12:11:21 EDT
The avenues available to regular Java programs are available in Eclipse.  In a 
standard Java scenario, you would either have these third party jars in your 
ext dir or explicitly listed on the classpath.  Out of the box Eclipse does not 
consider the ext classloader or the app classloader.  bug 30919 comment 20 
indicates how users can get Eclipse to behave in exactly the same way.  

What I was trying to say earlier was that either you have components/modules or 
you don't.  If you do, use plugins and the plugins contain their constituent 
pieces.  If you don't, you are in regular Java-land and should use whatever 
facilities are available to you there.
Comment 19 Ben Youngdahl CLA 2004-06-08 17:53:56 EDT
> What I was trying to say earlier was that either 
> you have components/modules or you don't.  If you
> do, use plugins and the plugins contain their constituent 
> pieces.  If you don't, you are in regular Java-land 
> and should use whatever facilities are available to you there.

Let's assume for the moment that a scenario exists where a person wants to
deliver a plug-in that cannot, for whatever reason, contain a JAR it is
dependent on.  That is the "if you don't" scenario at the end of your comment.

Please help me understand how I can have my plug-in use "regular Java-land
facilities" such as adding new JARs to the classpath *without*:
  1. affecting the Eclipse startup commandline parameters, 
  2. writing my own classloader, or
  3. forcing the dependent JARs to be installed in the JVM's "ext" folder
Those are the limitations I think for most people requesting this
bug/enhancement be addressed.

I see a disconnect -- doing what I'm asking is not a "regular Java-land
facility" scenario.  Eclipse is an RCP *framework* -- it's not quite the same as
running the framework hosted app by itself from the command line.  Eclipse, in
my opinion, needs to be more flexible about allowing plug-ins to load classes
from other places on the filesystem.  

I'd hate to see this become a philisophical discussion.  A lot of solid,
real-world scenarios have been raised in these comments about cases where this
kind of support is necessary.

Just my 2 cents.

Comment 20 Ben Youngdahl CLA 2004-06-08 18:13:21 EDT
I'm going to try to play around with classloaders in Eclipse and see if I can't
find a way around this.
Comment 21 Jeff McAffer CLA 2004-06-08 22:44:27 EDT
Question:  If you were not using Eclipse, how would you get access to these 
third party jars that are installed in random locations?

I am postulating that whatever mechanism you would use there is available to 
you when using Eclipse.

Question(s):  Assume you have plugin org.foo.P 1.2.4 and it lists
   $someplace$/p.jar
on its classpath.

- Who installed p.jar? (and when relative to installing P)
- How do you know that code in p.jar on your disk matches the 
expectations/published API of P 1.2.4? 
- How is P updated?  
- Who chooses $someplace$ and who sets the value?
- How does this play at development time (both for P and those who depend on P)
- How does this play at build time (is p.jar built by us?  Where is the source? 
Is it available for others to compile against?)
- More generally, what are the possible external visibilities of the code in 
p.jar?

Actually doing the change to enable this would be relatively trivial but 
finding good answers to these and the other questions is hard.

BTW, for the interested reader, look at EclipseClassLoader.findClassPathEntry
().  In there we substitution for $ws$, $os$ and $nl$ as well as constructing 
the dev mode classpath.  All of the above are considered to be relative to the 
plugin itself.  

You'll have to create a new kind of BundleFile to reference a file that is not 
part of a bundle but once that is in place, just add another case at the bottom 
to resolve your var to the appropriate BundleFile, create a ClasspathEntry, add 
it to the list and watch the classes load.

<of course, I've not done this myself and it is in no way endorsed by the 
Eclipse team>
Comment 22 Ben Youngdahl CLA 2004-06-08 23:31:55 EDT
> Question:  If you were not using Eclipse, how would 
> you get access to these third party jars that are 
> installed in random locations?

I would think most likely by either specifying them on the classpath of the VM
at startup time, installing them in the "ext" portion of the VM, or using one's
own class loader.  I realize that all of these options are probably available
with Eclipse, but I don't think they map well to what I thought this RFE was
requesting.

Eclipse is a "multi-app" ("plugins") RCP framework, however, so I don't see
using the VM classpath as appropriate, putting things in the "ext" part of the
VM also seems inappropriate, and I'm still looking into how I could hook things
up via my own classloader.

The rest of your questions raise interesting issues, which of course have no
universal, easy answers, but let me take a stab at some context:

> Who installed p.jar? (and when relative to installing P)

Assume p.jar pre-existed on the filesystem before P was installed, similar to
say a p.jar installed as an "ext" extension to the VM.

> How do you know that code in p.jar on your disk matches the 
expectations/published API of P 1.2.4? 

The same way you know that the API of the VM or an extension installed in "ext"
matches the expectations of P 1.2.4.  Ie. assume we don't expect Eclipse to
solve this problem.

> How is P updated?

Let's say the update manager.  p.jar is updated external to Eclipse.

> Who chooses $someplace$ and who sets the value?

Perhaps definable within the plugin.xml (for a fairly static environment), an
environment variable, or dynamically at plug-in start-up (for a more dynamic
environment).

> How does this play at development time (both for P and those who depend on P)

I believe Eclipse already allows a means for external JARs to be made available
at compilation time.  It's the runtime that seems more limited.

> How does this play at build time (is p.jar built by us?  Where is the source?
Is it available for others to compile against?)

Let's say that p.jar is not built by Eclipse, the source is not available, and
it is avaible (as .classes) for others to compile against.

> More generally, what are the possible external visibilities of the code in p.jar?

Unlimited.  p.jar exists as an entity outside the control of Eclipse.  Eclipse
gives up being the center of the universe as far as p.jar is concerned.

Comment 23 Pascal Rapicault CLA 2004-06-09 09:17:11 EDT
If I'm correct what you want is actually what is proposed as enhancement in bug 
#63350. It will give you the same kind of thing that you are looking for.

However this does not solve the questions that Jeff raised earlier regarding a 
solid tooling where people that fetch code from a repo have to worry about the 
location of possibly missing external jars, etc.
Comment 24 Christophe Avare CLA 2004-06-09 12:51:23 EDT
Thanks everybody for your answers, as my problem (at least) is solved.
The lastest trends in the dicussion are of great interest because they are
basically related to the effort that should be made to make Eclipse as pervasive
as possible.
While all this discussion must be moved at some point into another more
appropriate place, i bet that people interested in this topic have a better
understanding of some issues to develop some ideas:

1/ Java Classpath vs Eclipse plugins
From the very beginning the classpath mechanism have been far from satisfactory,
especially on windows platforms.
The Eclipse plugin based framework is of great interest as a centralizing
mechanism: we can compare its advantages to the assembly concept for .NET which
is a huge improvement over the so-called DL-Hell.
But the outside world is both hostile and not very responsive to change, so
making the Eclipse a success leads to compromises, some of them being exposed in
this discussion.

2/ Making Eclipse attractive
While eradicating diversity in the distribution of Java components or
applications may require a lot of energy and time, it may be possible to make
the platform more attractive by considering a new project aimed at converting en
masse all the well known librairies used by the Java Community.
Consider the potential boost to the Eclipse community if all the Java librairies
from Apache and other famous Open Source be available as plugins from a central
repository.
I'm sure that all the legal issues can be overcomed and the community is ready
to contribute to the repository.
Comment 25 John Arthorne CLA 2005-01-06 12:25:31 EST
Marking old enhancement requests that we don't intend to fix in the near future
as RESOLVED LATER. This does not mean we will never do them, but it means we
have no specific plans to address them.
Comment 26 John Arthorne CLA 2005-01-12 13:26:37 EST
*** Bug 14020 has been marked as a duplicate of this bug. ***
Comment 27 John Arthorne CLA 2005-01-12 13:26:46 EST
*** Bug 19327 has been marked as a duplicate of this bug. ***
Comment 28 John Arthorne CLA 2009-08-18 16:13:36 EDT
[LATER->WONTFIX] The "LATER" bugzilla resolution is being removed so reopening to mark as WONTFIX.
Comment 29 John Arthorne CLA 2009-08-18 16:19:40 EDT
[LATER->WONTFIX] The "LATER" bugzilla resolution is being removed so reopening to mark as WONTFIX.
Comment 30 Thomas Watson CLA 2009-08-18 17:39:34 EDT
Note that Equinox does have the ability to add "external" library jars to the Bundle-ClassPath with the "external:" prefix.  But it still has the issues outlined in comment 21.