Bug 288700 - [Build path] Problems due to automatic adding/removal of dependent libraries in Eclipse 3.5
Summary: [Build path] Problems due to automatic adding/removal of dependent libraries ...
Status: VERIFIED WONTFIX
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.5   Edit
Hardware: All All
: P3 normal with 1 vote (vote)
Target Milestone: 3.6 M7   Edit
Assignee: Jay Arthanareeswaran CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-09-05 15:26 EDT by Matthias Basler CLA
Modified: 2010-04-26 10:47 EDT (History)
6 users (show)

See Also:


Attachments
JAR that contains a Class-Path: ../.. entry (479 bytes, application/octet-stream)
2010-03-19 21:28 EDT, Felipe Leme CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Basler CLA 2009-09-05 15:26:34 EDT
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729)
Build Identifier: I20090611-1540

Since Eclipse 3.5, when I add a dependency (like a local jar file), Eclipse will look up its dependencies from its manifest and add them to the build path too (if found in the same folder I guess).
This is currently causing several negative side effects:

1. Setting the source file/folder for the automatically added libraries mixes up the sources, with the result that source code lookup will almost never work. This is already covered by issue 281551.
2. When having a library A (from folder AA) on the build path and adding library B (from folder BB) to the build path, the library A from folder BB will be added again (if existing there), so the build path shows two identical libraries, just with different paths.
(I cannot even imagine what will happen if the two libraries were different versions!)
3. Having added a library with several (automatically added) dependent libs, I decide to replace on of the dependent libs by another version, maybe for testing compatibility. Therefore I click on the library and choose "Remove from build path". However, Eclipse will not remove this single lib (as expected), nor the lib with its dependencies ... it will remove the whole hierarchy, that is, the manually added library with all its dependencies. So I have a "all or nothing" situation, which is inacceptable.
Eclipse should at least(!) ask the user and offer the possibility to retain the rest of the libs.
4. It is not possible to tell from the package explorer which libraries were manually vs. automatically added.
5. It is confusing that the automatically added libs appear in the package explorer, but not in f.e. the build path or user libraries dialogs.

Solutions imho:
a) Remove this "feature".
b) Make it optional for Eclipse to automatically add/remove dependent libraries to/from build path (probably the best solution)
c) Redesign this feature to avoid the mentioned problems.

See following related issues:
- Issue 281551: Source attachments wrong for automatically added libraries 
- Issue 283806: New API is needed to attach source code for referenced jars in the Class-Path: entry


Reproducible: Always

Steps to Reproduce:
Should be easy to reproduce from the what I told in "Details". Should you still need sth., let me know.

A good example to play with is the GeoTools library (having a deep hierarchy of libs in one folder)
http://sourceforge.net/projects/geotools/files/GeoTools%202.5%20Releases/2.5.7/geotools-2.5.7-bin.zip/download



I'm filing it as a bug (not enhancement), since for me it is a regression that prevents me from seriously working with Eclipse 3.5. No workaround is known to me.

(P.S. My general opinion: Eclipse may assist the user, but the user must always have the last word.)
Comment 1 Jay Arthanareeswaran CLA 2009-09-23 05:50:41 EDT
All points mentioned in comment #0 except points 2 and 3  seem to have been covered/discussed in the other bugs (bug 281551, bug 252431 and bug 283806)

Talking about the ability to remove a single referenced jar from the build path,  question arises on whether one can also add it back.

Dani/Olivier,
What do you think?
Comment 2 Dani Megert CLA 2009-09-23 11:53:44 EDT
>Talking about the ability to remove a single referenced jar from the build
>path,  question arises on whether one can also add it back.
Yes, definitely but we probably don't want to go down that path. I'd say allowing to specify on the first-class JAR whether this feature is desired is the right approach for this bug here.
Comment 3 Jay Arthanareeswaran CLA 2010-02-01 09:40:35 EST
(In reply to comment #1)
> Talking about the ability to remove a single referenced jar from the build
> path,  question arises on whether one can also add it back.

Refer to bug 252431 comment 17 - The API will support to add(or remove) either all or none of the chained entries and not for the individual chained jars.
Comment 4 Dani Megert CLA 2010-02-12 08:05:27 EST
Thinking of this more, I no longer think we should allow to disable it: given you've put a JAR on your build path that requires the other ones then those should be added if present. I agree that when coming from 3.4 where we didn't support this, you could have some misconfigured build path, but cleaning that up takes not more time than going to check a checkbox, assuming that the source attachment issues (bug 283806 and bug 281551) are fixed.

Of course removing a referenced library from the build path should not remove the parent - actually 'Remove from Build Path' should be disabled for referenced entries.
Comment 5 Matthias Basler CLA 2010-02-12 10:55:51 EST
Hi Dani and others,

Dani wrote:
[...] actually 'Remove from Build Path' should be disabled for
referenced entries.

You see this from the "we want a clean built path" perspective, but this is only one way of seeing.
Please re-consider scenario 3 of my original bug report and tell me how to do this if it isn't possible to remove/replace an automatically added jar by another version from, lets say, another directory in the file system in order to check if the new version of the library is still compatible with the application and other libraries.

To put it very bluntly:
For testing reasons I sometimes WANT an incomplete or custom build path, and if Eclipse doesn't allow this easily any more, it means a lot of extra work for me, like moving libraries from one directory to another just so that Eclipse automatically finds exactly the libraries that *I* want it to find.

This is why I think this new behaviour is - under certain circumstances - a regression and should be optional.
I hope you understand my point of view.
Comment 6 Markus Keller CLA 2010-02-16 05:07:00 EST
> To put it very bluntly:
> For testing reasons I sometimes WANT an incomplete or custom build path

Hmm, that may work at compile time, but how do you test this? Doesn't the VM *always* resolve the entries in Class-Path? Even if we allow you not to see dependencies on the build path, they will still be there when you put the JAR on the classpath and launch with the Java executable. Your only choice is to put the versions you want to test at the right places on the filesystem.

Or you could try to put the new version of the referenced JAR in front of the other JAR, but then you will have both on the classpath, which is also not the situation you really want to test. And that's already possible in 3.5.
Comment 7 Olivier Thomann CLA 2010-02-16 13:54:19 EST
(In reply to comment #5)
> To put it very bluntly:
> For testing reasons I sometimes WANT an incomplete or custom build path, and if
> Eclipse doesn't allow this easily any more, it means a lot of extra work for
> me, like moving libraries from one directory to another just so that Eclipse
> automatically finds exactly the libraries that *I* want it to find.
To be blunt: what is the use case for this ?

From comment 0,
1) This will be taken care of using a new API that will enable referenced jars to have their own source and javadoc attachments (see bug 252431).
2) I think this should be done during the resolved classpath. If the entries resolve to the same file, it should not appear twice.
3) The replacement should be added before the library that references the entry you want to replace.
4) This should be fixed by using the new API since it will be possible to know whether an entry is a referenced entry or not.
Comment 8 Matthias Basler CLA 2010-02-17 14:31:31 EST
(In reply to comment #7)
> > For testing reasons I sometimes WANT an incomplete or custom build path
> To be blunt: what is the use case for this ?

I'll try to give an example of both:

Prerequisite:
I have, on one harddisk, several different libraries in several versions, because I require them for different projects. Among them are large libraries like GeoTools which has far over 100 files in its libs folder - for each version.

Since I am not particularily fond of the idea of having multiple copies of all these libraries and dependency JARs all over my HDD, I link the jars or parts of them from these "shared remote locations" to the projects "as needed" whenever this is possible.

Scenario 1:
I develop an application using a library which has some optional dependencies (for support of different databases, PDF rendering etc.). 
At some point I decide to check whether my application can do without a specific one of these (maybe optional) jars, because I don't need a specific functionality and care about file size. Therefore I want to remove this specific jar (not all the jars from the library) from the build path and check if/where I get compile or runtime errors. 

Scenario 2:
I develop an application the Geotools library at version 2.4
GeoTools 2.4 comes (among several others) with JTS (Java Topology Suite) at version 1.8. At some time I realize that I want to use some features of JTS that are available only at version 1.10. So I want to try and check if JTS 1.10 is backward compatible and can be used with GT 2.4 in my application as well. I'd now remove the JTS jar and its dependencies from the build path (without removing the dozen of other jars) and add the JTS 1.10 jar and its new dependencies. Then I could test whether everything plays well together.

With Eclipse 3.5 this would require me to copy all the required files into a project specific library folder and add them from there to the build path. 

I wonder if my way of organizing libraries is really so exceptional...

---
In reply to comment 6:
> Hmm, that may work at compile time, but how do you test this?
> Doesn't the VM *always* resolve the entries in Class-Path?

I am rather sure that only the JARs specified as the project build path belong to the build path, not all the JARs that happen to be in the same directories as these JARS. Note that my build path is a list of JARs, not a list of directories where the VM can use whatever it finds.
Feel free to tell me if I am wrong.
Comment 9 Dani Megert CLA 2010-02-18 03:14:02 EST
Instead of modifying the build path of the project where the JAR with required ones is, you can simply add a required project where you do the juggling.

I don't buy the "having multiple copies" argument: you can have one single copy and either link it into the projects where it is needed or add it there as external archive to the build path when needed. This is like the JRE which you also don't have to copy for each project.

>I wonder if my way of organizing libraries is really so exceptional...
I guess so, since no one else complained so far ;-)

NOTE: I filed bug 303155  for the remove issue.


Let's hear what Markus and Olivier have to say. My vote is to close this as WONTFIX.
Comment 10 Matthias Basler CLA 2010-02-18 12:32:33 EST
(In reply to comment #9)
> Instead of modifying the build path of the project where the JAR with required
> ones is, you can simply add a required project where you do the juggling.

Looks like a misunderstanding here. Let me clarify:
1. I download and extract, lets say, GeoTools 2.4 library (+ deps) into folder A
2. I download and extract JTS 1.10 library (+ deps) into folder B
I do not (and do not intend to) touch any of these files.

3. I have my Eclipse workspace in a folder C.
I want to add selected JARS from folder A and folder B to the build path of a project in C as external achives. But I cannot add the JARS from location B because similar JARS (usually of a different version) are already in folder A and they were previously added automatically to the project when I selected some JARs from folder A.

> I don't buy the "having multiple copies" argument: you can have one single copy
> and either link it into the projects where it is needed or add it there as
> external archive to the build path when needed. This is like the JRE which you
> also don't have to copy for each project.

Adding them as external archive is exactly what I have done all the time.
Comment 11 Markus Keller CLA 2010-02-18 13:06:43 EST
(In reply to comment #8)

> Scenario 1:
[..]
> Therefore I want to remove this
> specific jar (not all the jars from the library) from the build path and check
> if/where I get compile or runtime errors. 
> 
> Scenario 2:
[..]
> Then I could test whether everything plays well together.

I agree that the automatic addition of required JARs does not allow you any more to see compile errors the way you could see them in 3.5 when you removed a dependency and then added another one.

But also in 3.5, you could *not* test what you saw in the Package Explorer, since it's the VM that resolves the "Class-Path:" entries in the JARs on the classpath. Since you cannot stop the VM from doing this, you could not produce the runtime situation that matches the build path in Eclipse. Even if you added the replacement JAR for a required JAR at the beginning of the build path, the required JAR will still be resolved and used at run time.

> I am rather sure that only the JARs specified as the project build path belong
> to the build path, not all the JARs that happen to be in the same directories
> as these JARS. Note that my build path is a list of JARs, not a list of
> directories where the VM can use whatever it finds.
> Feel free to tell me if I am wrong.

You're right that only the specific JARs on the Eclipse build path will be passed when the VM is launched, but the VM does not care about the classpath when resolving JARs from the "Class-Path:" attribute. Those are always resolved relative to the location of the original JAR (on the filesystem), and the VM will load them from there.

We should not allow something at compile time which we cannot support at run time, so we should not add an option to suppress required JARs.
Comment 12 Olivier Thomann CLA 2010-02-18 13:15:49 EST
(In reply to comment #11)
> We should not allow something at compile time which we cannot support at run
> time, so we should not add an option to suppress required JARs.
I concur with this as this is a completely artificial situation that doesn't reflect a possible configuration at runtime.
Comment 13 Matthias Basler CLA 2010-02-18 15:18:45 EST
Well, if I am the only having problems with this situation, you may close this bug. I am sure I can find a workaround, and as a last resort I still have a working Eclipse 3.4 at hand for these specific situations.
Oh yes, and thanks for clarification about this VM behaviour in comment 11, Markus. This was important to understand.
Comment 14 Felipe Leme CLA 2010-03-19 21:28:19 EDT
Created attachment 162584 [details]
JAR that contains a Class-Path: ../.. entry

If a lib has a Class-Path: entry with an invalid value (../.. in my case), it crashes the project with the following exception:

Problems occurred when invoking code from plug-in: "org.eclipse.jface".

org.eclipse.core.runtime.AssertionFailedException: null argument:
	at org.eclipse.core.runtime.Assert.isNotNull(Assert.java:85)
	at org.eclipse.core.runtime.Assert.isNotNull(Assert.java:73)
	at org.eclipse.core.runtime.Path.initialize(Path.java:577)
	at org.eclipse.core.runtime.Path.<init>(Path.java:185)
	at org.eclipse.core.internal.resources.WorkspaceRoot.getProject(WorkspaceRoot.java:179)
	at org.eclipse.jdt.internal.core.ExternalFoldersManager.isExternalFolderPath(ExternalFoldersManager.java:77)
	at org.eclipse.jdt.internal.core.JavaProject.addToResult(JavaProject.java:2634)
	at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:2611)
	at org.eclipse.jdt.internal.core.JavaProject.resolveClasspath(JavaProject.java:2662)
	at org.eclipse.jdt.internal.core.JavaProject.getResolvedClasspath(JavaProject.java:1861)
	at org.eclipse.jdt.internal.core.PackageFragmentRoot.getRawClasspathEntry(PackageFragmentRoot.java:547)
	at org.eclipse.jdt.internal.ui.viewsupport.JavaElementLabelComposer.appendVariableLabel(JavaElementLabelComposer.java:1076)
	at org.eclipse.jdt.internal.ui.viewsupport.JavaElementLabelComposer.appendPackageFragmentRootLabel(JavaElementLabelComposer.java:1058)
	at org.eclipse.jdt.internal.ui.viewsupport.JavaElementLabelComposer.appendElementLabel(JavaElementLabelComposer.java:263)
	at org.eclipse.jdt.ui.JavaElementLabels.getElementLabel(JavaElementLabels.java:521)
	at org.eclipse.jdt.ui.JavaElementLabels.getStyledElementLabel(JavaElementLabels.java:496)
	at org.eclipse.jdt.ui.JavaElementLabels.getStyledTextLabel(JavaElementLabels.java:424)
	at org.eclipse.jdt.internal.ui.viewsupport.JavaUILabelProvider.getStyledText(JavaUILabelProvider.java:178)
	at org.eclipse.jdt.internal.ui.packageview.PackageExplorerLabelProvider.getStyledText(PackageExplorerLabelProvider.java:70)
	at org.eclipse.jface.viewers.DelegatingStyledCellLabelProvider.getStyledText(DelegatingStyledCellLabelProvider.java:195)
	at org.eclipse.jface.viewers.DecoratingStyledCellLabelProvider.getStyledText(DecoratingStyledCellLabelProvider.java:192)
	at org.eclipse.jface.viewers.DelegatingStyledCellLabelProvider.update(DelegatingStyledCellLabelProvider.java:103)
	at org.eclipse.jface.viewers.DecoratingStyledCellLabelProvider.update(DecoratingStyledCellLabelProvider.java:134)
	at org.eclipse.jface.viewers.ViewerColumn.refresh(ViewerColumn.java:145)
	at org.eclipse.jface.viewers.AbstractTreeViewer.doUpdateItem(AbstractTreeViewer.java:932)
	at org.eclipse.jface.viewers.AbstractTreeViewer$UpdateItemSafeRunnable.run(AbstractTreeViewer.java:102)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.core.runtime.Platform.run(Platform.java:888)
	at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:48)
	at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
	at org.eclipse.jface.viewers.AbstractTreeViewer.doUpdateItem(AbstractTreeViewer.java:1012)
	at org.eclipse.jface.viewers.StructuredViewer$UpdateItemSafeRunnable.run(StructuredViewer.java:481)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
	at org.eclipse.core.runtime.Platform.run(Platform.java:888)
	at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:48)
	at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:175)
	at org.eclipse.jface.viewers.StructuredViewer.updateItem(StructuredViewer.java:2111)
	at org.eclipse.jface.viewers.AbstractTreeViewer.createTreeItem(AbstractTreeViewer.java:827)
	at org.eclipse.jface.viewers.AbstractTreeViewer$1.run(AbstractTreeViewer.java:802)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
	at org.eclipse.jface.viewers.AbstractTreeViewer.createChildren(AbstractTreeViewer.java:776)
	at org.eclipse.jface.viewers.TreeViewer.createChildren(TreeViewer.java:639)
	at org.eclipse.jface.viewers.AbstractTreeViewer.handleTreeExpand(AbstractTreeViewer.java:1442)
	at org.eclipse.jface.viewers.TreeViewer.handleTreeExpand(TreeViewer.java:937)
	at org.eclipse.jface.viewers.AbstractTreeViewer$4.treeExpanded(AbstractTreeViewer.java:1453)
	at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:126)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1176)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1200)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1185)
	at org.eclipse.swt.widgets.Tree.gtk_test_expand_row(Tree.java:2062)
	at org.eclipse.swt.widgets.Widget.windowProc(Widget.java:1601)
	at org.eclipse.swt.widgets.Display.windowProc(Display.java:4197)
	at org.eclipse.swt.internal.gtk.OS._gtk_main_do_event(Native Method)
	at org.eclipse.swt.internal.gtk.OS.gtk_main_do_event(OS.java:7586)
	at org.eclipse.swt.widgets.Display.eventProc(Display.java:1185)
	at org.eclipse.swt.internal.gtk.OS._g_main_context_iteration(Native Method)
	at org.eclipse.swt.internal.gtk.OS.g_main_context_iteration(OS.java:1858)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3110)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2405)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2369)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2221)
	at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:493)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:113)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:194)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:368)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:559)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:514)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1311)


I tried to create a .zip with a buggy jar, but the problem didn't happen when I imported it. But it can be reproduced by using the attached jar and following the steps below:

1.Create a new java project
2.Outside eclipse, add baddie.jar to the project's directory
3.Refresh the project on Eclipse
4.Add baddie.jar to the classpath. If it doesn't complain right away, try to refresh the project.


I know that this particular JAR has an invalid entry and it should be fixed, but sometimes it's out of our control, and Eclipse should just issue a warning reporting the problem (rather than making the whole project unusable).
Comment 15 Jay Arthanareeswaran CLA 2010-03-20 02:02:53 EDT
I am looking in to the reported failure. And I think it can be moved to a separate bug.
Comment 16 Felipe Leme CLA 2010-03-22 12:31:12 EDT
I agree, I just posted it here initially to avoid duplication (as I wasn't sure).
Comment 17 Jay Arthanareeswaran CLA 2010-04-06 03:08:44 EDT
(In reply to comment #14)
> Created an attachment (id=162584) [details]
> JAR that contains a Class-Path: ../.. entry
> ...
This is being tracked under bug 308150.
Comment 18 Jay Arthanareeswaran CLA 2010-04-19 03:17:52 EDT
No plan to fix this.
Comment 19 Jay Arthanareeswaran CLA 2010-04-19 03:22:02 EDT
Oops, forgot to set the target.
Comment 20 Satyam Kandula CLA 2010-04-26 10:42:55 EDT
Verified for 3.6M7 using build I20100424-2000
Comment 21 Jay Arthanareeswaran CLA 2010-04-26 10:47:45 EDT
Verified.