Bug 44215 - Running out of memory when compiling.
Summary: Running out of memory when compiling.
Status: RESOLVED FIXED
Alias: None
Product: AspectJ
Classification: Tools
Component: Compiler (show other bugs)
Version: 1.2.1   Edit
Hardware: PC Windows 2000
: P1 critical (vote)
Target Milestone: 1.5.3   Edit
Assignee: Andrew Clement CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 40764 76097 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-10-06 06:46 EDT by Gerard Fisher CLA
Modified: 2007-03-02 17:19 EST (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gerard Fisher CLA 2003-10-06 06:46:58 EDT
Eclipse IDE: v2.1.1 Build 200306271545

Eclipse Feature: org.eclipse.aspectj_1.1.4

Eclipse Plugins: org.aspectj.ajde_1.1.4, org.eclipse.adjt.ui_0.6.4, 
org.eclipse.aspectj_1.1.4

CPU: Intel Celeron 1200MHz; RAM 512 MB

-------------------------------------------------------------------------------

AspectJ Error: OutOfMemoryError thrown: null

I have installed AspectJ and got it to compile, weave and run correctly on 
simple demo programs. However, compilation halts partway on a large project 
with 14.6 MB source code. I have searched Eclipse environment but see no way of 
increasing RAM available during compilation.

Cheers.
Comment 1 Peter Kalmus CLA 2003-10-17 12:40:08 EDT
Shouldn't this be highest priority?

To increase available memory for eclipse, add the following as an argument to 
eclipse.exe:

-vmargs -Xmx256M
Comment 2 Brett Kotch CLA 2003-10-17 12:42:12 EDT
At by the defination of Severity, shouldn't this be critical? 

Critical is defined as 
   "crashes, loss of data, severe memory leak"

Comment 3 Mik Kersten CLA 2003-10-20 01:50:27 EDT
The OutOfMemory error should be resolved by increasing the VM max that Eclipse 
is launched with, e.g.: 

eclipse.exe -data C:\Dev\workspace -vmArgs -Xmx384M

Keep increasing memory until you don't run out, nad note that the AspectJ 
compiler uses more than the JDT compiler (as noted in the AspectJ FAQ).  Also 
note that the "vmArgs" comes last (I'm not sure if this is still important, 
but it has been in past version of Eclipse).  

If this doesn't solve the problem please reopen the bug.
Comment 4 Peter Kalmus CLA 2003-10-20 10:34:03 EDT
Mik, this is a partial workaround (partial, because you still must restart 
eclipse after n ajc compiles, n depending on size of project, n = 3 typical in 
my experience for -Xmx384M).

Which is great, but I'm wondering:
  + fine that ajc uses lots of memory.  Shouldn't the memory be released back 
to Eclipse *after* the compile?  That's the bug.

  + is it going to be fixed?
 
  + if it's not fixed, then this bug should be open and CRITICAL.  I've used 
1.1.4 and the memory is *not* freed after an ajc compile, so unless I'm 
missing something, this bug should be open.
Comment 5 Adrian Colyer CLA 2003-10-20 11:02:21 EDT
Let's reopen. We know that there are problems in this area (whether in AJDT or 
in the underlying AspectJ support). We also know that this is a high priority 
item in our development plan. A quick scan revealed no other open bug that is 
tracking this issue.
Comment 6 Mik Kersten CLA 2003-10-20 11:21:36 EDT
Peter: memory not being freed is definitely a bug.  Since AJDT holds on to a 
structure model between compiles, it is possible for it to appear that memory 
use keeps rising through the first few compiles.  I haven't profiled AJDT for 
a while, but after 1/2 dozen compiles or so the memory use should stabilize 
and not increase.  

Let's keep this open until we figure out what's going on.  I will reassign to 
Andy who has worked on AJDT memory issues in the past.  In the meantime I 
suggest that you try raising your vm max even higher to test whether the 
memory use stabilizes.
Comment 7 Matt Chapman CLA 2004-11-18 04:27:32 EST
*** Bug 76097 has been marked as a duplicate of this bug. ***
Comment 8 Matt Chapman CLA 2004-11-18 09:11:07 EST
Here are the results of my attempts to measure the memory usage of AJDT and
AspectJ. I did try using Hyades, but only managed to get results from very small
projects, and then couldnt derive much meaningful info from the results. Instead
I  kept running eclipse (in self-host mode) with different JVM heap sizes, using
binary search to determine the smallest possible heap size. My test involves the
compilation (doing a clean project and rebuild operation each time) of one
project, which is the org.eclipse.ajdt.ui plugin. The dependent plugins were
made available in the same workspace as binary plugins. The plugin has around
47,000 lines of code, with an aspect that gets woven in 100 places. 

Heap size           Av. compile time

256m                     25s
117m                     62s
116m                     out of memory

Next, I compiled the same project from an ant build using the iajc task with
fork=true and emacssym=true (to generate the structure model):

Heap size           Av. compile time

256m                     24s
71m                      56s
70m                      out of memory

(without building the structure model, a heap of 59m was sufficient).

The timings are rather approx. as this is about memory usage not performance -
all they show is that in a constrained memory environment performance degrades,
no doubt caused by heap fragmentation and increased GC cycles.

So, 117m is required in the eclipse case, and 71m in the standalone case. This
leaves 46m for eclipse & AJDT. I then measured the heap sized required to start
eclipse in the java perspective with an open editor, both with and without the
ajdt plugins active. The approximate breakdown of memory usage can then be
attributed something like this:

Total:             117m
ajc:                71m
eclipse startup     26m
ajdt startup:       11m
ajdt/eclipse other:  9m

Also, in the eclipse case where there was only just enough memory to complete
the project compilation, I repeated the clean/compilation cycle numerous times,
and this never resulted in an out of memory error. This imples there isn't a
noticable memory leak, at least in the functionality exercised.

This was all done with AJDT build 1.2.0.20041117183203.

So unless anyone can show otherwise, my conclusion is that there isn't a memory
leak in current builds, and that the memory usage of AJDT itself is not overly
excessive. The memory usage of ajc during compilation (plus the building of the
structure model) is fairly high, and according to Andy this is something the
AspectJ team intend looking at, so I've passing this bug over for that purpose.
Comment 9 John Carlson CLA 2005-04-21 01:20:41 EDT
Would be nice if this could be fixed.  I am trying to use ajc without eclipse,
and it is running out of memory.  I tried both -vmargs and -vmArgs and
neither were accepted.

John
Comment 10 Andrew Clement CLA 2005-04-21 03:40:32 EDT
John - if using ajc outside of eclipse, you need to modify the batch file that
launches the compiler to ensure the JVM runs with adequate memory - you can't
change it with -vmargs, that is only works for launching eclipse.

Open the ajc.bat file in the aspectj<version>/bin directory and change the
default memory specified, the default is -Xmx64M which gives the compiler only
64M, change that to 256M.
Comment 11 Alan Williamson CLA 2005-08-16 17:59:31 EDT
> So unless anyone can show otherwise, my conclusion is that there isn't a memory
> leak in current builds, and that the memory usage of AJDT itself is not overly
> excessive. 

Not overly excessive?  You are kidding yeah?  I have given up with Eclipse now
as within 3 builds i am out of memory and have to restart.  I have since
developed an ant script to do this for me on the command line; it still uses a
huge amount of memory but it means i don't have to restart Eclipse every 10 minutes.

But what concerns me is this.  I have had to set my "maxmem" to 384MB to
successfully weave my single aspect into a JAR file the size of 2MB!!!  Surely
that can't be right?   What if I want to weave bigger JAR's?  Will the memory
increase accordingly or is there a minimum amount of memory it needs to get
'warmed' up?
Comment 12 Matt Chapman CLA 2005-08-17 05:41:13 EDT
Alan, my comment related to only the AJDT side of the picture - i.e. my
conclusion was that the memory usage of just AJDT was fairly small compared to
the memory usage of the AspectJ compiler (particular in incremental mode). Hence
this bug is against the compiler, as the best overall improvement can be
achieved by focusing on the memory usage of the compiler - I believe Andy has
some ideas under development in this area.
Comment 13 Alan Williamson CLA 2005-08-17 06:38:54 EDT
(In reply to comment #12)
> Alan, my comment related to only the AJDT side of the picture - i.e. my
> conclusion was that the memory usage of just AJDT was fairly small compared to
> the memory usage of the AspectJ compiler (particular in incremental mode). Hence
> this bug is against the compiler, as the best overall improvement can be
> achieved by focusing on the memory usage of the compiler - I believe Andy has
> some ideas under development in this area.

Okay thanks for the clarification Matt.  

Maybe, the technique of paging out to disk should be considered instead of using
memory.  I know it will be slower, but it would allow the compiler to scale to
very large projects without having to hit this memory issue.   Most developers
have oodles of disk space at their disposal.
Comment 14 Matt Chapman CLA 2005-08-18 08:36:10 EDT
*** Bug 40764 has been marked as a duplicate of this bug. ***
Comment 15 Andrew Clement CLA 2006-03-27 03:42:11 EST
Under bug 128650 the memory usage of AspectJ/AJDT as of now has been reduced by 50% - this applies to all projects I have tested.
Comment 16 Radek Wikturna CLA 2006-06-22 09:04:21 EDT
I'm completely new to AspectJ but nevertheless, to see an OutOfMemory error the very first time I tried using it is very disappointing.

Eclipse
  Version: 3.2.0
  Build id: I20060217-1115
  run with 'eclipse.exe -vmargs -Xmx512m'
AspectJ 
  ajdt_1.4.0.20060622064301

run on a bigger project (cca 5000 Java files, 30MB on disk, + using plenty of other Jars)
-> when I ran 'convert to AspectJ' the compilation ended with OutOfMemoryError (512M used and not released, CPU 100%, I had to restart Elipse, when this has happend several times in a row I gave up :-((
Comment 17 Andrew Clement CLA 2006-07-28 06:53:46 EDT
I've just committed the code for 146781 - which switches AspectJ to a pipelining compiler, drastically reducing memory requirements for compilation.  this will be in an AJ dev build and AJDT build shortly.
Comment 18 Andrew Clement CLA 2006-08-08 09:05:50 EDT
should be much improved in recent builds - closing this bug.  We can track any new problems under a new bug.
Comment 19 Leher Devaiah CLA 2007-03-02 17:19:49 EST
Despite setting the memory to 896m this error persists. I have no problem building in a cmd prompt using ant.