Bug 29668 - Ant classloader persists after run
Summary: Ant classloader persists after run
Status: RESOLVED WORKSFORME
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Ant (show other bugs)
Version: 2.1   Edit
Hardware: PC Windows NT
: P3 normal (vote)
Target Milestone: 2.1 RC3   Edit
Assignee: Darin Swanson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 34863 98717 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-01-16 15:54 EST by Matthew Conway CLA
Modified: 2006-01-19 01:09 EST (History)
2 users (show)

See Also:


Attachments
Project containing failure case (2.56 KB, application/octet-stream)
2003-01-16 15:55 EST, Matthew Conway CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew Conway CLA 2003-01-16 15:54:27 EST
If I run an ant script using eclipse, classloaders created in that script to
load other jars stick around past the completion of the script, thereby causing
the jars to stay locked.

Import the attached project, and run the build.xml twice - the first time, it
runs ok, the second, it fails to delete jar.jar (pun intended =)

If you run a 3rd time, it does works again, indicating the classloader does get
cleared at some point.  However, if I make the classloader a static in
LockJar.java, and add tasks.jar to ant runtime classloader instead, then things
stay locked.

This is basically what my custom task is doing - it has a static singleton which
contains a classloader.

Clicking Restore Defaults in ant runtime prefs seems to clear the lock - but
this is annoying when you have a number of jars in there which need to be re-added.
Comment 1 Matthew Conway CLA 2003-01-16 15:55:41 EST
Created attachment 3017 [details]
Project containing failure case
Comment 2 Darin Swanson CLA 2003-02-05 17:43:55 EST
Investigate if we can clean this up or is it simply a case of waiting for the 
gc?
Comment 3 Darin Swanson CLA 2003-02-18 12:44:15 EST
When I modified the Lock.jar to have the static var containing the classes 
classloader (and even before this modification), the garbage collection took 
its time cleaning up the classloader. Sometimes the garbage collection only 
seemed to be prompted when I actually went to inspect reference graph for the 
classloader (using Optimizeit Profiler 4.2).
I could never get the build to fail twice in a row.

This is with some tweaks we have been doing with the AntClassLoader and the 
AntSecurityManager so I will double check these haven't change this behavior.
Comment 4 Darin Swanson CLA 2003-02-18 16:43:05 EST
On 20030218, I cannot reproduce the problem. Sometimes the build will fail. But 
if I try again it works.
Profiling, the AntClassLoader is always gc'd eventually.
Comment 5 Matthew Conway CLA 2003-03-03 13:04:21 EST
This is still failing for me on a recent build (200302270800 but with a cvs
build of externaltools/ant.core from today, 20030303)

If you get the ProcessExplorer utility:
http://www.systernals.com/ntw2k/freeware/procexp.shtml

It will show you which processes have a lock on which file.
Run the "all" target for build.xml for the previously attached project once,
then run processex, and do a find (ctrl-f) for "jar.jar".  You'll see that the
java.exe that is running eclipse has a lock on that jar.

If you run build.xml again (this time use the init target so it doesn't delete
the jar), you'll see that even though we have done a new build, and supposedly
have gc'ed the classloader, processexp still reports a lock being held on jar.jar.

If you keep doing the "all" target, at some point the lock goes away, and I'm
not sure why (probably has something to do with the file being deleted, but if
there is a lock on the file, I don't know understand why it is allowed to be
deleted)

As a workaround, it is possible to force a garbage collection, and the lock will
be released.  To force the GC, I use the Memory Monitor View plugin availbale here:
http://mmoebius.gmxhome.de/eclipse/resources.htm#mem_mon

Is it poissible to force a GC after build completion?
Comment 6 Matthew Conway CLA 2003-03-04 09:07:11 EST
Actually, while GCing works in the simple test case attached, it does not always
work.  The files stay locked when I run a build using our build infrastructure,
but I haven't isolated how it is different from the simple case.  When I do so,
I'll provide another test case, providing you haven't fixed the problem by then ;-)
Comment 7 Darin Swanson CLA 2003-03-06 14:26:29 EST
Matt I think what you are seeing the the differences between normal GC'ing 
and "classloader" GC'ing...from what I have seen (and by talking to others) 
they definately are birds of a different color.  There actually is no guarantee 
that a classloader will get gc (this from the spec).
Could your build infrastructure be tagging into the tasks that end up "leaking" 
so that they cannot be GC'd and therefore the classloader cannot be GC'd?
Comment 8 Matthew Conway CLA 2003-03-10 12:03:52 EST
The problem with my build infrastructure ended up being due to task replacing
System.out/err/in and System.setSecurityManager() - Since the instances of these
  objects set in these system slots were loaded by the ant classloader, there
ended up being a reference to that classloader beyond the life of the ant run.  

However, even when I cleaned up that code and verified that it worked in a CLI
ant run (by using BuildListener.buildFinished to pause the run and check things
out) - I still get a leak when running in eclipse - so maybe eclipse is somehow
preserving my instances by grabbing one of the System members or somthing.

The javascript task in ant seems to do the same thing (i.e. CLI ant with a noop
javascript task causes a leak, remove the task, and leak goes away)

I've given up on trying to solve this for now - this is one of those things that
would be a non issue if ant was run as a separate process instead of within
eclipse's JVM.  What are the major stumbling blocks for making this happen?

Comment 9 Darin Swanson CLA 2003-03-10 12:15:25 EST
>What are the major stumbling blocks for making this happen?
History: we have to maintain the current and develop the "new" functionality
Time: we ran out of it

The Eclipse specific tasks would not work at all...they require the plugin 
classloaders of their plugins to work properly.
All extension point tasks / types / extra classpath entries would pretty much 
have to be ignored.

Resolution of "Java"...currently the external tools / ant framework has no 
knowledge / requirements of the Java infrastructure.

Some type of socket communication between the VMs to handle showing the output 
with decent linking and coloring.

The good news I can provide is that this is my first priority for the next go 
around.
Comment 10 Darin Swanson CLA 2003-03-16 19:41:54 EST
*** Bug 34863 has been marked as a duplicate of this bug. ***
Comment 11 Darin Swanson CLA 2006-01-19 01:09:41 EST
*** Bug 98717 has been marked as a duplicate of this bug. ***