Bug 116489 - [modulecore] More ways to specify external jars
Summary: [modulecore] More ways to specify external jars
Status: NEW
Alias: None
Product: WTP Java EE Tools
Classification: WebTools
Component: jst.j2ee (show other bugs)
Version: 1.0   Edit
Hardware: PC Windows XP
: P3 enhancement with 3 votes (vote)
Target Milestone: Future   Edit
Assignee: Kaloyan Raev CLA
QA Contact: Chuck Bridgham CLA
URL:
Whiteboard: ProjectStructure
Keywords: helpwanted
Depends on:
Blocks:
 
Reported: 2005-11-15 14:54 EST by Igor Fedorenko CLA
Modified: 2010-01-28 11:59 EST (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Igor Fedorenko CLA 2005-11-15 14:54:54 EST
Our codebase is a mixture of plain java, OSGi and J2EE components andas most of
the products nowadays we use quite a lot of thirdparty libraries. We wrap each
thirdparty JAR or related set of JARs in a bundle project, similar to how
Eclipse.org wraps JUnit, ANT or AXIS and make our projects depend on these
wrappers. With WTP however we cannot do that and have to either copy the JARs to
WEB-INF/lib directory and use "External jars" from within workspace.

Ideally, I am looking for direct support of "JAR wrapper" projects as depenent
modules but ability to pick external JARs from workspace will do too.
Comment 1 Darryl Miles CLA 2005-11-16 14:48:35 EST
Just to clarify, your "wrapped JARs" are also refactored to a new package
location, but they exists as JARs (which may not be Eclipse projects themselves)
outside of the workspace tree ?

I don't think the "wrapped nature" of the JARs affects your underlying feature
request?


I too am interested in your feature request.

The facility I am looking for is to allow the "Web App Libraries [projectname]"
tree to allow the insertion of "linked JARs", as well as the current real copied
JARs it finds in WEB-INF/lib after a refresh.  Maybe the JAR icon can be marked
on the UI to indicate linked from real.

I would like these linked JARs to allow both absolute file path to a JAR and
references to User-Libraries JAR working set, so that I could insert a
reference, this means I only need to maintain one User-Libraries list across all
my projects.

For each of these linked JARs during the publish or export process they would be
copied to the WEB-INF/lib directory but the point being that I no longer need to
physically put a copy of that JAR into my project's WEB-INF/lib as I do now.


Maybe there is overlap here with the Java Build Path -> Order and Export.  Its
not clear exactly what ticking anything other than a Java Source folder should
do in the grand scheme of things.

Also the only way to get more items into the Order and Export is to include them
in the class path.  But this is not the same as a linked JARs.  The way I think
about it class path dependancies are dependancies that are outside the scope of
the deployed application.  They are libraries that the web-app expects to be
available at runtime but the application itself does not provide.

The linked JARs however are dependancies that are inside the scope of the
deployed application which the application will provide in WEB-INF/lib.



Ultimatly this heads in the direction of being able to create an EAR with a WAR
inside that and a jst.utility inside that and the jst.utility uses Hibernate, it
should be possible to export it up the chain but into the resuling WAR/EAR but
have Eclipse auto-cancel multiple references to the same real file the linked
JAR defined.

If my feature request differs vastly from your problem please say so and I'll
open another request.

My 2 cents...
Comment 2 Igor Fedorenko CLA 2005-11-16 15:47:51 EST
It is already possible to "link" JARs from *outside* of workspace to both
WEB-INF/lib and EAR. Although I personally do not find User-Libraries that
useful, being able to use them for WAR/EAR dependent unitility JARs looks like a
valid enhancement request to me.

Having all that said, this is not the problem I am looking to solve as in my
case all JARs are located *inside* workspace. For example, my workspace has
org.apache.log4j (java) project that has log4j.jar exported JAR file on
classpath. Ideally, I want to be able to link all exported classpath entries of
org.apache.log4j project to WEB-INF/lib but as I mentioned in original
description linking with log4j.jar from org.apache.org project will do too.

Workspace-relative references decouple my WAR/EAR configuration from physical
filesystem which is especially useful when many developers work on the same
codebase -- each developer can create workspace anywhere he likes and can even
have projects located outside of workspace physical directory tree.  Refernces
to "all classpath entries exported by a project" reduces coupling even further
as structure of org.apache.log4j can now be changed without having to updated
every project that uses it.
Comment 3 Darryl Miles CLA 2005-11-24 01:12:54 EST
After a small exchange of emails (between Igor and myself) over a week ago it was agreed that time that Igor was a little confused over over what Eclipse does  and does not do right now.


The cleanest way for me to describe what we both want is to read my comment #1 but revise my 5th paragraph from comment #1, to read like this:

I would like these linked JARs to allow:
* relative file path to a JAR inside the workspace and (added item)
* absolute file path to a JAR anywhere on the computer and
* references to User-Libraries JAR working set, so that I could insert a reference, this means I only need to maintain one User-Libraries list across all my projects.


It might read a whole lot clearer if everytime I used the term "Linked JARs" you replaced it with "J2EE Dependancy Linked JAR" which is not mean the same as a "Classpath Dependancy Linked JAR" which is a feature we already have available to us when we configure the build path.

A "J2EE Dependancy Linked JAR" would:
 * be available to the compiler to compile the project (they would always have a higher preceedence than classpath dependancies, since in real life when deployed this is also the case)
 * get copied into WEB-INF/lib from wherever it was linked from, but ONLY at deployment assembly time, this means only during server publishing and export WAR and export WAR into EAR operations.
 * They could be visualised by the devloper by expanding the "Web App Libraries [myproject]" tree and seeing linked references or named library workeding sets under this tree (not under the project tree, those are classpath dependancies)
 * The Project -> J2EE Dependancies -> Web Modules properties page could have its buttons updated to configiure ONLY J2EE Dependancies Linked JAR (never classpath dependancies)
 * A right click action could be introduced on the "Web App Libraries [myproject]" to allow the insertion of new J2EE Dependancies Linked JAR from that point also and a "remove" action for to take out those in the list.



I'm supprised Igor has not responsed or cleaned up his comment #2 as reading it would only confuses anyone on what exactly he wants.  It is not already possible to do what Igor expected Eclipse to do and our feature requests do have synergy.

I have also opened up bug #116856 to better explain the confusion in the J2EE dependancies properties dialog as it is now (M8/M9) in relationship to what people are expecting it to do for them.  They are expecting the behaviour of this bug report is trying to describe.  But at the moment it simply reconfigures the build classpath not the WEF-INF/lib contents during deployment assembly.  Which is what people expect just like it can do for J2EE Utility JARs.


I would appreciate feedback from Igor and Chuck just to confirm I'm not confusing everyone further.  It is somewhat difficult to describe something that does not yet exist to another party through bugzilla.

Thanks for your patience.
Comment 4 Gabriele Garuglieri CLA 2005-11-25 03:07:57 EST
I absolutely agree with comment #3.
The way it is now the "Web App Libraries [projectname]" is almost useless for us.

We download project jars from  a maven like internal library server (not all our machine have internet access) and we divide them in compile time and run time jars that we keep in dirs separate from web content.
For us phisycally putting jars in /WEB-INF/lib is a no-no. I'm rather tired to cleanup jars checked in our cvs repository by error by inexperienced contractors.

Quoting from previous comment, the  "Web App Libraries [projectname]" jars should "get copied into WEB-INF/lib from wherever it was linked from, but ONLY at deployment assembly time, this means only during server publishing and export WAR and export WAR into EAR perations." and the copy should be in the assembly location, NOT within source tree.
Comment 5 Igor Fedorenko CLA 2005-11-28 14:18:03 EST
Wow, that's the first time somebody (mis) quoted me! :-) 

Ok, here are my updated requirements:

Extend WAR and EAR projects to allow additional ways to reference JAR files. In particular,
* ability to reference JAR files from within workspace
* ability to use JDT classpath containers or something similar to reference collection of JAR files within or outside of workspace (this should cover "User library" request from comment #2 too)
Such referenced JARs should be used during compilation (let me stress it again, by *reference*, nothing should be copied during compilation). Such referenced JARs should also become part of logical WAR/EAR layout and be properly exported during WAR/EAR export and be properly deployed into test servers. During deployment, JAR files should be copied only if test server requires so (in other words, you need to support servers that are able to load JARs from outside of WAR/EAR directory structure).

Hope this is clear enough. 

I also have some ideas about how I would implement this, so let me know if you are interested ;-)
Comment 6 Darryl Miles CLA 2005-11-30 16:08:36 EST
Sorry to have misquoted you, please correct me accordingly.


Igor wrote:

> During deployment, JAR files should be copied only if test server
> requires so (in other words, you need to support servers that are
> able to load JARs from outside of WAR/EAR directory structure).

Can you explain that server runtime feature.

It is my understanding that by definition an application container's job is to "contain the environment" it runs in, part of that containment means you can't load JARs from outside the WAR/EAR directory structure via the normal mechanisms.  Its not the correct way of doing things to hardwire an absolute path to the JAR outside of the WAR/EAR directory structure the whole point of an WAR/EAR file is thats its a 100% contained deployable unit.

Now it is possible for some containers to allow you to see extra locations available to the application's runtime classpath, like Tomcat's $CATALINA_BASE/shared and $CATALINA_HOME/common as well as the JRE classpath.

But these should be treated as classic "Classpath Dependancies" and Eclipse can already deal with them in that way, just add those libraries to the Build Path and away you go.


The "only if test server requires so" sounds something too complex for a baseline WTP platform to support, since there is no mechnism to discover the information necessary to do that across all local/remote runtimes.


I agree with all the other parts of your comment, just not the bit I quoted.
Comment 7 Igor Fedorenko CLA 2005-11-30 16:46:57 EST
This is a little trick I originally wrote for JBoss (way before WTP became available) and now ported to WTP/Tomcat. Here is the basic idea:

Instead of deploying real WAR/EAR file WTP deploys a map file that describes location of various parts of the WAR/EAR. App server gets extended to know how to read and deploy such "mapped" WAR/EAR.

This saves a lot of packaging and copying during publishing (read: publish is virtually instantaneous) and in case of Tomcat solves problems with JAR file locking described in bug 103391.
Comment 8 Darryl Miles CLA 2005-11-30 18:06:42 EST
Now I understand your line of thoughts relating to your bespoke plugin, I have no exposure to WIN32 platform issues.

The logic I was questioning sounds like something your runtime plugin should take care of after it is handed the J2EE WAR that has been virtually assembled in memory.

So what are you asking the platform to do that you dont think should be done by your plugin, or put another way; what new things can the platform do to help your plugin that you think other plugins would also benifit from sharing the use of.  i.e. why should that logic be in the platform and not your plugin ?

For me on Unix there is no jar locking issue and the amount of copying is only relevant during the first publish cycle (when no files exist) after that point in time incremental/delta changes are made which are instant.
Comment 9 Kaloyan Raev CLA 2008-09-01 12:07:20 EDT
Assigning to Kiril for evaluation