Community
Participate
Working Groups
From the ant-dev@jakarta.apache.org: >>While playing with Eclipse 2.0.1 IDE and ant 1.5 plugin, I noticed >>that Eclipse (or ant) didn't release memory used. >>And after 10 to 20 use of ant, the IDE shutdown itself.
Created attachment 2168 [details] buildfile that leaks 20000k
Running the attached buildfile leaves the eclipse java process about 20000k larger than before the build (according to windows' unreliable task manager). It only takes me 3 or 4 runs to either get an out of memory error (and it is NOT pretty when Eclipse runs out of memory saving your workspace!) or for the process to simply exit (presumably out of memory). My source is 1600+ files, but the javac is forked so that shouldn't matter too much.
I'm going to second this one. I'm also using "-vmargs -Xmx256M" on a machine with 1G of memory and I periodically run out of memory at which point Eclipse 2.0.1 unceremoniously shuts down. For a long time I didn't know what was causing the problem, but over time I have come to believe that Ant is involved. If I use the project build, the problem never seems to surface; if I use Ant (which I'm doing more frequently these days), it shows up regularly. For the record, I'm also using the AntView plugin but don't believe it is responsible as I have seen the same problem using Eclipse Ant. My Ant version is 1.5 if it makes any difference.
I have done some looking for leaks on the 2.1 code base and I am not seeing much. Would either of you be able to check out the integration/M2 builds that come out this week and report back?
Using the link Darin sent me for the M2 build (Build ID 200210181211) I was able to successfully compile a dozen times. The Windows task manager showed memory getting reclaimed occasionally; the most eclipse went up to was 99k; it generally wavered between 28k and 40k. In the previous build I never saw any evidence of garbage collection kick in. I'm betting this one is OBE.
Moving to M4...good to keep checking.
From the newsgroup (david_lee@ulstek.com.tw) > To start with, I am using the M4 build. Java VM is j2sdk1.4.1_01. > > What my project does is to build ejb 1.1 files using the XDoclet API and > Ant. > > As for the bug report, I have found that it has already been filed at > Bugzilla Bug 24448. > > Previously, with the 1.3.1 vm, the ide freezes after 15 to 20 builds. Now > with the 1.4 vm, the ide runs out of memory at 40 to 50 builds. I am also > enclosing a doc file with the messages that I am seeing. > > For reproducing the test case, I would recommend downloading the Lomboz > Eclipse plug-in, create a J2EE project and an ejb container. What the > plug-in does is just to help in writing the XDoclet - Ant xml files. Then > execute the "Generate EJB classes" function repeatedly or write a simple > build.xml file which calls the Lomboz generated Ant scripts and execute it > using the built in ant task. The results would be the same ..... Out of > Memory Error.
I seem to be getting the same error with 2.1 M4
Does anyone know if this memory leak has been addressed in the 2.2 version?? If so, I can download it even before the M1 release, because it's quite annoying ANT-ing from within a Command Prompt. Thx.
I can vouch that this bug is still present in the 200212181304 build (M4 build). I have noticed this issue through any number of build files, even for simple tasks. I run on Win2k with Eclipse args of : -vmargs -Xmx256M My machine has 512MB of memory and with the task manager going, I generally die when Eclipse hits 185MB+
I noticed something today - when running ANT over and over, I steady consumed more memory. At one point I was up to about 120MB. About 5 minutes later I was ready to run ANT again, and happily noticed I had shot down to only 51MB (I figure some sort of GC had happened). When I ran ANT, the memory usage shot immediately back up to around 145MB (which is about where it would have been after running ANT again, had the GC not occurred).
*** Bug 29421 has been marked as a duplicate of this bug. ***
I`ve found a workaround for it: Class AntCorePlugin from plugin org.eclipse.core.ant: . . . public class AntCorePlugin extends Plugin { private static AntClassLoader antClassLoader;
Created attachment 3364 [details] AntCorePlugin
I`ve found a workaround for it: Class AntCorePlugin from plugin org.eclipse.core.ant is attached. This means that Eclipse has to be restarted whenever there are changes to the Ant classpath in the preferences. But thats much better then getting OutOfMemory exceptions.
Looking at the patch, you have added caching of the AntClassLoader instead of recreating the class loader for each build. In fact previous to M4 we did exactly that. This was identified as a memory leak. Some Ant tasks that execute leak memory and there is no way to guarantee that they don't. Without creating a new classloader every time we could not protect Eclipse from this problem. So unfortunately, I do not believe this is a solution but thanks anyways.
*** Bug 29961 has been marked as a duplicate of this bug. ***
This is the only serious issue I have found with 2.1 M4 and 2.1 M5/RC0 is this leaking of mem. I run eclipse on Windows XP with the command line "eclipse.exe -vm <path to 1.4.1 vm> -data <path to workspace>" and by using ant internally I get about 23 invocations before my process is reported as being 18xMB and I die. Actually most times the IDE zombies and I have to manually kill it.
Comment #17 This means it's better having all ant tasks leaking instead of some? How about fixing the leaking ant tasks instead of trying to fix them with a workaround in eclipse?
It would be great if you worked with Apache Ant to identify and help them fix any tasks that might be leaking.
How about using the solution from tomcat. In the Tomcat plugin has all we need. PluginClassloaderWrapper WebAppClassloader a.s.o. Yes, a classloader without leaking. :-)
How about executing ant as an external process? This might also provide for another feature which I would be interested in, namely the ability to use different Java versions for different ANT Builds. Sorry if this has already been suggested and shot down.
Definately not shot down...we just ran out of time. But it is currently possible..we just don't provide all the support that we would like to. Please see bug 24129.
Ulrich, can you provide details on how the tomcat classloaders can guarantee no leakage. I am no expert but I do not think that this is possible. As an example some of the leaks we are seeing occur because of Threads being registered for Runtime shutdown hooks etc
On build I20030218, using Optimizeit Profiler 4.2 enterprise edition I am having trouble finding many cases where we are leaking. The case I have found is running the xdoclet.ejb.EjbDocletTask task. A reference to the org.apache.log4j.LogManager class remains after the build. A static variable of this class pretty much holds on to the rest of the Ant world. Optimizeit is not showing any instances of LogManager remaining and does not show references to classes (at least not that I can find). Can the community help shed some light on this problem? Can anyone provide a different test case? The builds I am trying, the org.eclipse.ant.internal.core.AntClassLoader is getting garbage collected. This is the classloader we use to load the Ant classes used for the build.
As well see http://nagoya.apache.org/bugzilla/show_bug.cgi?id=17216 (Execute task and the ProcessDestoryer memory leak).
Well this may be the thing for me. I didn't notice a memory leak until I started using xdoclet which was around the same time I started with RC0. I just create a 2nd ant build file which just does an exec against the 1st build file and I can still see my memory going up but at a reduced rate. Previously it would go up at approx. 6MB per execution and not it seems to only be 1.2MB per execution. This 1.2MB may be related to the ProcessDestoryer memory leak previously memtioned.
The list of tasks which will exhibit the ProcessDestroyer memory leak is long. javac (default compiler adapter and fork=true, Jikes adapter) javadoc cvs cvspass cvstagdiff cvschangelog exec junit (forked) ... at least 15 more
I have an ant jar which patches the ProcessDestroyer memory leak (as far as I can tell). I can attach it if anyone is interested.
Released some changes to the Eclipse integration build listeners (AntProcessBuildListener and ProgressBuildListener) to null out as much state as possible so a little bit more can be gc'd
Created attachment 3718 [details] "kit" to produce patched Ant 1.5.1 for ProcessDestroyer The patch I submitted to Ant for the ProcessDestroyer leak was accepted for the 1.6 milestone (see comment #27 or http://nagoya.apache.org/bugzilla/show_bug.cgi?id=17216). I am attaching a zip which includes a build script and the patched ProcessDestroyer which can be used against an existing ant 1.5.1 distribution to produce a patched ant 1.5.1 jar. To create a patched 1.5.1 using the zip, simply extract the zip to a directory and run ant in that directory. The ant script requires "ant.home" to be set to the ant home directory. It will produce an ant-patch.jar in that directory. The patch is created by extracting the classes from the Ant 1.5.1 release and replacing only the ProcessDestroyer class file. A new jar is created (ant-patch) using the existing Ant 1.5.1 class files and the new ProcessDestroyer class files. An attribute is added to the manifest indicating that it is a patched ant jar.
I patched the ant.jar in eclipse 2.1 M5/RC0 but I am still experiencing the memory leaks causing eclipse to shutdown when I run numerous Ant builds that execute XDoclet tasks. Could someone confirm whether or not this patch has fixed the problem for them.
The ProcessDestroyer patch will not fix the leak(s) that I was profiling when executing a build that used the xdoclet.ejb.EjbDocletTask task.
*** Bug 33391 has been marked as a duplicate of this bug. ***
Marking as won't fix but really it is can't fix. The solution / workaround is for us to implement bug 24129.
I just wanted to say that after evaluating eclipse for our workplace, we have decided not to switch largely because of this bug. With large projects, importing a dozen or so jars, it only takes half an hour or so for an outofmemoryexception to kill the workspace. This is after setting the heap to 256meg or larger. I don't mean to belittle the great work done on eclipse, I hope things come togethor in the future.
A workaround seems to be to invoke ant as an external program. I have setup external launchers with cmd.exe as the tool and passing ags of '/c "ant ejbdoclet ..." and the memory usage is orders better.
Yes this will work as you running Ant in a separate vm which terminates on completion of the build. See comment #24 and bug 24129 for further clarification and examples.
*** Bug 33906 has been marked as a duplicate of this bug. ***
*** Bug 32336 has been marked as a duplicate of this bug. ***
*** Bug 35503 has been marked as a duplicate of this bug. ***
*** Bug 35751 has been marked as a duplicate of this bug. ***
*** Bug 35864 has been marked as a duplicate of this bug. ***
*** Bug 36072 has been marked as a duplicate of this bug. ***
*** Bug 36048 has been marked as a duplicate of this bug. ***
Hi, I've tried to integrate the Classloader from the tomcat project unfortunatly this does not help. Anyway you may find it usefull that the objectstyle team relesead a patched org.eclipse.core.ant_2.1.0 plugin.You can download it from http://sourceforge.net/project/ showfiles.php?group_id=50507. Ulrich
Thanks for the information Ulrich, but do you know what the patch entails? The release notes for the patch make no indication of what has been changed.
The patch includes the AntCorePlugin attachment from this bugreport. We can't wait until all memory holes are fixed in ant, the jvm or whatever. With Eclipse 2.1 it's not possible to use ant. Please do me a favour and look at it from the user point of view. What is more important: 1. Good looking code in Eclipse but no ant support. 10 runs until there is no more memory for the classloader and the vm halts. Or imagine a user with 10 projects. The unlucky developer don't even get a chance to build his projects. Okay it's possible to increase the memory size which brings us to 2 rebuild all. 2. A hack that enables the ant usage. Same classloader for each ant run and a day were you can do the job payed for. There are so many people with headaches about that. Don't get me wrong I don't have the intention to offend you. I have the feeling that this is more a problem of priority rather then a technical problem. As far as I understand it, the AntCorePlugin set's up a new ClassLoader for each ant run to avoid memory leaks. That is a goog thing. But the fix results in other more problems. Ulrich
Does someone have a list of ant tasks known to leak memory? I know that almost all of the tasks that launch an external process leak memory. See Comment #29. With the patch of the ProcessDestroyer leak, I can run all of my build files without leaking memory. None of my custom tasks leak memory, and none of the core tasks or optional tasks that I run leak. I think the solution that keeps the ClassLoader resident in memory will still lead to frustrating memory leaks, albeit a *lot* slower. I prefer the external tool-> Program solution, see bug 24129.
*** Bug 36639 has been marked as a duplicate of this bug. ***
*** Bug 36678 has been marked as a duplicate of this bug. ***
*** Bug 36781 has been marked as a duplicate of this bug. ***
*** Bug 36827 has been marked as a duplicate of this bug. ***
The external tool stuff won't help in my case. I launch Ant programatically. The external tools don't provide a public api. I totally agree with you that the resident classloader still leaks. A crash after 12 hours is much better then one after 2 minutes. Ulrich
Ulrich, can you open a separate bug with your usecase for being able to programatically designate reuse of the AntClassLoader. Thanks.
*** Bug 36812 has been marked as a duplicate of this bug. ***
Just a note that an alpha version of integrated support for launching Ant in a separate VM is provided in the latest integration builds. See bug 24129 for more details.
*** Bug 37616 has been marked as a duplicate of this bug. ***
*** Bug 38338 has been marked as a duplicate of this bug. ***
*** Bug 37912 has been marked as a duplicate of this bug. ***
*** Bug 38990 has been marked as a duplicate of this bug. ***
*** Bug 39118 has been marked as a duplicate of this bug. ***
*** Bug 40250 has been marked as a duplicate of this bug. ***
*** Bug 41646 has been marked as a duplicate of this bug. ***
*** Bug 45424 has been marked as a duplicate of this bug. ***
*** Bug 45889 has been marked as a duplicate of this bug. ***
*** Bug 45948 has been marked as a duplicate of this bug. ***
*** Bug 46218 has been marked as a duplicate of this bug. ***
*** Bug 46790 has been marked as a duplicate of this bug. ***
People, this really sucks! Do you expect me to read all this crap to check what I have to do to fix the problem MYSELF with a "workaround"? I expect to read a short solution and not dozens of postings! And I don't expect to apply a patch which don't work (build error). I simply want a patched file and you to tell me where to copy it. Please! This is a bad bug handling which is done here!
This one should probably be marked 'fixed in 3.0' since 3.0m5 defaults to run ant in its own VM now.
The outstanding XDoclet issue: http://opensource.atlassian.com/projects/xdoclet/secure/ViewIssue.jspa?key=XDT- 419
Generally this is "fixed" in 3.0 with the fixes in the Apache Ant code base and Eclipse defaulting to run Ant builds in a separate VM. Changing subject so maybe people will find this bug instead of logging duplicates.
*** Bug 53318 has been marked as a duplicate of this bug. ***
*** Bug 58075 has been marked as a duplicate of this bug. ***
*** Bug 62489 has been marked as a duplicate of this bug. ***
*** Bug 86523 has been marked as a duplicate of this bug. ***
Has the bug been fixed ? Thanks
I am getting similar errors to those described in this bug with Eclipse 3.1.1. That version of eclipse is using Ant 1.6.5 so it should not be leaking. I am not running ant in a seperate JVM. Have all measures been taken to ensure that ant doesn't leak?
> I am getting similar errors to those described in this bug with Eclipse 3.1.1. > That version of eclipse is using Ant 1.6.5 so it should not be leaking. I am > not running ant in a seperate JVM. Have all measures been taken to ensure that > ant doesn't leak? This is a very hard question for the Eclipse Ant integration to answer. I have done extensive testing to ensure that our integration implementation does not leak. Can I test all Ant tasks and infrastructure to ensure that it does not leak? The answer is an obvious no. Any Ant build with its likely unique series of tasks run in the same JRE as Eclipse has the potential to leak especially when most tasks make the assumption that they will execute once and then the VM will exit. What I would suggest is that the community take the time to profile the build that is suspected of leaking and report the bug(s) to the responsible party...be it Eclipse or Ant. The second best alternative is to log a reproducible bug report with test case and buildfile so I can determine exactly where the leak is. Note this should be a new bug report and not an addition to this behemoth :-)
Well, this is coming back to bite us again as well. We are using AntRunner programmatically to run an Ant-based builder based on CppTasks. We use the platform runtime only version of Eclipse as the starting point for our IDE (we do not want the added bloat of the JDT - which Ant UI requires), so we do not have access to the Ant UI stuff. This means, I also do not have the option of running the Ant builder in a separate VM without a HUGE amount of work and code duplication. Anyway, if you run repeated Ant builds the platform will eventually run out of memory and crash. I have tried to profile this, but I must admit, I don't really know what I am doing. I am trying the YourKit Java profiler, and I have managed to confirm that the leak is in fact the org.eclipse.ant.internal.core.AntClassLoader class. I took a memory snapshot, ran 5 builds in a row, and lo and behold there are 5 instances of AntClassLoader kicking around, each referencing megabytes of other objects. I tried to profile with object allocation turned on, but every profiler I have tried simply crashes when you try this. I think this has to do with the way AntRunner mucks with classloaders or something, because it always gets into the build then just crashes violently. Anyway - I have no idea where to look next. Can look at the retained instances of AntClassLoader and see that the classes Vector contains 641 Object references. Good luck figuring out which one is at fault... So, I have exported a dump of the AntClassLoader object in HTML form and I'll attach it to this bug, but I can't make heads or tails of it. I have no idea how to determine what is at fault here. Since I am dealing with AntRunner programmatically, is there any way to explicitly force an object to be deleted? I am setting my AntRunner class to null when I am finished, but obviously that isn't helping. There are no instances of AntRunner left in my profiling snapshots, so it is getting cleaned up properly. What the heck is holding onto the AntClassLoader then? <sigh> I wish I knew more about this stuff...
Created attachment 34091 [details] Dump of AntClassLoader object
Thanks Mark for the information. Any class that is leaked and was loaded by a particular class loader will cause the class loader to not be able to be GC'd. I will look at the dump but needle in a haystack does come to mind. What would really allow me to get to the bottom of this is to log a new bug report with a buildfile, that when run repeatedly as a same JRE build causes an OOM failure. Then I can profile to find who and what is leaking.
I can try to minimize this down to a simple build file, and I'll file a new bug if I do. That could be difficult however, because we have our own builder calling AntRunner, which in turn uses CppTasks, as well as our own extensions to CppTasks. It will get a little complicated. If I had to guess I would say CppTasks is the culprit. I have found numerous bugs in that package, and I am rapidly trying to figure out an exit plan in the back of my mind to get away from using it at all, now that we have shipped products based on it. ;o)
