Bug 316635 - Out of heap space while exporting
Summary: Out of heap space while exporting
Status: RESOLVED FIXED
Alias: None
Product: AJDT
Classification: Tools
Component: Core (show other bugs)
Version: 2.0.2   Edit
Hardware: PC Windows 7
: P3 critical (vote)
Target Milestone: 2.1.0   Edit
Assignee: AJDT-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-06-11 13:00 EDT by yoav CLA
Modified: 2010-08-26 16:57 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description yoav CLA 2010-06-11 13:00:40 EDT
java.lang.OutOfMemoryError
at org.aspectj.util.FileUtil.readAsByteArray(FileUtil.java:1181)
at org.aspectj.ajdt.internal.core.builder.AjBuildManager.copyResourcesFromFile(AjBuildManager.java:529)
at org.aspectj.ajdt.internal.core.builder.AjBuildManager.copyResourcesToDestination(AjBuildManager.java:466)
at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performBuild(AjBuildManager.java:363)
at org.aspectj.ajdt.internal.core.builde ... ob.run(AutoBuildJob.java:238)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

Compile error: OutOfMemoryError thrown: Java heap space


AJDT version 2.0.2
Comment 1 Andrew Eisenberg CLA 2010-06-11 13:51:23 EDT
This looks like an AJDT problem, so changing products.

First, I would suggest starting Eclipse with more heap space and permgen size.  Add this to your arguments when starting:

-vmargs -Xmx768M -XX:PermSize=64M -XX:MaxPermSize=128M

If that doesn't work, you can bump Xmx up to 1024.  If you are still getting problems let me know and this may be a real bug.
Comment 2 yoav CLA 2010-06-11 16:23:53 EDT
I am already using 1024m in eclipse.ini:

-startup
plugins/org.eclipse.equinox.launcher_1.0.201.R35x_v20090715.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.0.200.v20090519
-product
org.eclipse.epp.package.java.product
--launcher.XXMaxPermSize
256M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx1024m
Comment 3 Andrew Eisenberg CLA 2010-06-11 16:56:04 EDT
Now it would be helpful to know some more about the project you are working on.  

* How big is it?
* How many aspects?
* What kinds of pointcuts are you using?  Do you have any pervasive ones (eg- execution(* *..*(..))?
* What kind of export operation are you doing (eg- PDE export, export jar)?

Any chance you send the project my way and I can try it out and see if I can reproduce locally?
Comment 4 yoav CLA 2010-06-11 18:02:32 EDT
I sent you the info to your e-mail.

(In reply to comment #3)
> Now it would be helpful to know some more about the project you are working on. 
> 
> * How big is it?
> * How many aspects?
> * What kinds of pointcuts are you using?  Do you have any pervasive ones (eg-
> execution(* *..*(..))?
> * What kind of export operation are you doing (eg- PDE export, export jar)?
> 
> Any chance you send the project my way and I can try it out and see if I can
> reproduce locally?
Comment 5 Andrew Eisenberg CLA 2010-06-16 18:07:20 EDT
I was not able to open the archive you sent me.  Can you please re-send the archive using a standard zip or gzip format?  Also, please only send the smallest project that reproduces the problem.

If this is an aspectj problem, I will forward this bug to the aspectj team.
Comment 6 yoav CLA 2010-07-05 15:57:26 EDT
Happens with the new AJDT version as well (2.1.0).
Comment 7 Andrew Eisenberg CLA 2010-08-05 18:47:53 EDT
There has been quite a bit of work recently related to reducing the memory being used by the weaver.   If you want to try this out, you can with the latest snapshot of AJDT.  See the last few comments of Bug 278496.  If you set the following two system properties:

aspectj.minimalModel=true
aspectj.typeDemotion=true

then the size of the weaver should decrease significantly over time.  Please let me know if you try this out.
Comment 8 yoav CLA 2010-08-06 08:36:55 EDT
Should it be added to eclipse.ini ?

(In reply to comment #7)
> There has been quite a bit of work recently related to reducing the memory
> being used by the weaver.   If you want to try this out, you can with the
> latest snapshot of AJDT.  See the last few comments of Bug 278496.  If you set
> the following two system properties:
> 
> aspectj.minimalModel=true
> aspectj.typeDemotion=true
> 
> then the size of the weaver should decrease significantly over time.  Please
> let me know if you try this out.
Comment 9 yoav CLA 2010-08-09 15:40:45 EDT
I added these lines to eclipse.ini :
-Daspectj.minimalModel=true
-Daspectj.typeDemotion=true

Is this the correct way to turn on these system properties?

(In reply to comment #8)
> Should it be added to eclipse.ini ?
> 
> (In reply to comment #7)
> > There has been quite a bit of work recently related to reducing the memory
> > being used by the weaver.   If you want to try this out, you can with the
> > latest snapshot of AJDT.  See the last few comments of Bug 278496.  If you set
> > the following two system properties:
> > 
> > aspectj.minimalModel=true
> > aspectj.typeDemotion=true
> > 
> > then the size of the weaver should decrease significantly over time.  Please
> > let me know if you try this out.
Comment 10 Andrew Eisenberg CLA 2010-08-09 17:26:30 EDT
(In reply to comment #9)
> I added these lines to eclipse.ini :
> -Daspectj.minimalModel=true
> -Daspectj.typeDemotion=true
> 
> Is this the correct way to turn on these system properties?

Looks right to me.  Any success?
Comment 11 yoav CLA 2010-08-10 01:48:04 EDT
No

Should I see a log print about this somewhere? Where?

(In reply to comment #10)
> (In reply to comment #9)
> > I added these lines to eclipse.ini :
> > -Daspectj.minimalModel=true
> > -Daspectj.typeDemotion=true
> > 
> > Is this the correct way to turn on these system properties?
> 
> Looks right to me.  Any success?
Comment 12 Andrew Eisenberg CLA 2010-08-10 12:03:17 EDT
Now that I think about it, these flags should be added to your build.properties file of the plugin(s) you are building.  And the minimal model flag is not necessary when running ajc headless (as you do for pde export).

In your build.properties, add the following two lines:

compilerAdapter.useLog=true
compilerArg=-vmargs -Daspectj.typeDemotion=true

And after building, in the build directory, you can find a @dot.log file, which will contain information about type demotion.  Actually, this file will only exist if you are doing your pde export from a headless mode, not if you are using an export wizard (in this case, Eclipse always cleans up all the build artifacts after the export).

Sorry for the confusion on my part, but please tell me if this works for you.
Comment 13 Andrew Eisenberg CLA 2010-08-10 12:14:29 EDT
And while you are fiddling with vmargs in the export, you may as well explicitly add a permsize and a max perm size.  Something like:

compilerArg=-vmargs -Xmx1024M -XX:PermSize=64M -XX:MaxPermSize=256M -Daspectj.typeDemotion=true
Comment 14 yoav CLA 2010-08-10 13:11:28 EDT
Sorry for the silly question, but where do I set these in Eclipse?
(In reply to comment #13)
> And while you are fiddling with vmargs in the export, you may as well
> explicitly add a permsize and a max perm size.  Something like:
> 
> compilerArg=-vmargs -Xmx1024M -XX:PermSize=64M -XX:MaxPermSize=256M
> -Daspectj.typeDemotion=true
Comment 15 Andrew Eisenberg CLA 2010-08-10 13:17:29 EDT
Inside of your build.properties file inside of your plugin.  Each plugin has a build.properties file that describes build information like source and bin directories, extra classpath entries, and compiler options.
Comment 16 yoav CLA 2010-08-10 13:48:33 EDT
I'm not sure what you refer to by 'plugin'. Is it the AJDT plug in? Or my project, which is not a plugin? where should the logfile be found?


(In reply to comment #15)
> Inside of your build.properties file inside of your plugin.  Each plugin has a
> build.properties file that describes build information like source and bin
> directories, extra classpath entries, and compiler options.
Comment 17 Andrew Eisenberg CLA 2010-08-10 14:17:30 EDT
Ugh...I'm mixing things up now.  Apologies.  You are not exporting a plugin, but exporting a regular AspectJ project as a jar file.  Is that right?  So, you can ignore what I said in the last few comments.

Let me look at things a little bit more.  I'll get back to you as to whether or not the particular system property is passed to the exporting call to ajc.
Comment 18 yoav CLA 2010-08-22 15:03:51 EDT
Were you able to find anything?

(In reply to comment #17)
> Ugh...I'm mixing things up now.  Apologies.  You are not exporting a plugin,
> but exporting a regular AspectJ project as a jar file.  Is that right?  So, you
> can ignore what I said in the last few comments.
> 
> Let me look at things a little bit more.  I'll get back to you as to whether or
> not the particular system property is passed to the exporting call to ajc.
Comment 19 Andrew Eisenberg CLA 2010-08-23 15:13:03 EDT
I am a bit surprised that you are not getting OOME when doing full builds, but you are when doing exports.  The export operation simply performs an incremental build of your project, so it should have the same memory requirements as builds at any other time.  Furthermore, the class files used for exporting are the class files in the output directory of your project.  In fact, according to my testing, the AspectJ compiler is never actually called if your output directories are up to date (ie- the code involved in the stack trace that you include below should not be reached if your bin folders are up to date).

My only suggestions to you are to keep using the minimalModel and typeDemotion system properties and also make sure that you have done a full build before running your export.
Comment 20 yoav CLA 2010-08-23 15:16:55 EDT
I am getting this errors with full builds. Actually on almost every full build I get it - I'm sorry I haven't been clear about this.

How do I turn this properties on for a regular Eclipse AJDT project?

(In reply to comment #19)
> I am a bit surprised that you are not getting OOME when doing full builds, but
> you are when doing exports.  The export operation simply performs an
> incremental build of your project, so it should have the same memory
> requirements as builds at any other time.  Furthermore, the class files used
> for exporting are the class files in the output directory of your project.  In
> fact, according to my testing, the AspectJ compiler is never actually called if
> your output directories are up to date (ie- the code involved in the stack
> trace that you include below should not be reached if your bin folders are up
> to date).
> 
> My only suggestions to you are to keep using the minimalModel and typeDemotion
> system properties and also make sure that you have done a full build before
> running your export.
Comment 21 Andrew Eisenberg CLA 2010-08-23 19:56:06 EDT
Are you on the latest dev build?  After that, all you need to do is add the options to the eclipse.ini.

You will know that this option is turned on if you see things like this in your system console:

ASPECTJ: aspectj.typeDemotion=true: type demotion switched ON
ASPECTJ: aspectj.minimalModel=true: minimal model switched ON

If you are on windows, then you need to run eclipsec.exe.  Also, using the -consoleLog argument is good regardless of your platform (add to eclipse.ini as well).
Comment 22 yoav CLA 2010-08-26 10:36:08 EDT
I do not see these debug prints in the console.
I am using:
Eclipse AspectJ Development Tools

Version: 2.1.1.e36x-20100817-1900
AspectJ version: 1.6.10.20100817163700

this is my eclipse.ini:
-startup
plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.0.v20100503
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
256M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx1024m
-Daspectj.minimalModel=true
-Daspectj.typeDemotion=true
-consoleLog

(In reply to comment #21)
> Are you on the latest dev build?  After that, all you need to do is add the
> options to the eclipse.ini.
> 
> You will know that this option is turned on if you see things like this in your
> system console:
> 
> ASPECTJ: aspectj.typeDemotion=true: type demotion switched ON
> ASPECTJ: aspectj.minimalModel=true: minimal model switched ON
> 
> If you are on windows, then you need to run eclipsec.exe.  Also, using the
> -consoleLog argument is good regardless of your platform (add to eclipse.ini as
> well).
Comment 23 yoav CLA 2010-08-26 10:37:28 EDT
I do not see these debug prints in the console.
I am using:
Eclipse AspectJ Development Tools

Version: 2.1.1.e36x-20100817-1900
AspectJ version: 1.6.10.20100817163700

this is my eclipse.ini:
-startup
plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.0.v20100503
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
256M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx1024m
-Daspectj.minimalModel=true
-Daspectj.typeDemotion=true
-consoleLog

(In reply to comment #21)
> Are you on the latest dev build?  After that, all you need to do is add the
> options to the eclipse.ini.
> 
> You will know that this option is turned on if you see things like this in your
> system console:
> 
> ASPECTJ: aspectj.typeDemotion=true: type demotion switched ON
> ASPECTJ: aspectj.minimalModel=true: minimal model switched ON
> 
> If you are on windows, then you need to run eclipsec.exe.  Also, using the
> -consoleLog argument is good regardless of your platform (add to eclipse.ini as
> well).
Comment 24 yoav CLA 2010-08-26 10:42:03 EDT
Ok, now I see these debug prints. I still get the out of memory exception when building. I am using:
Eclipse AspectJ Development Tools

Version: 2.1.1.e36x-20100817-1900
AspectJ version: 1.6.10.20100817163700

this is my eclipse.ini:
-startup
plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.0.v20100503
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
--launcher.XXMaxPermSize
256M
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx1024m
-Daspectj.minimalModel=true
-Daspectj.typeDemotion=true
-consoleLog

(In reply to comment #21)
> Are you on the latest dev build?  After that, all you need to do is add the
> options to the eclipse.ini.
> 
> You will know that this option is turned on if you see things like this in your
> system console:
> 
> ASPECTJ: aspectj.typeDemotion=true: type demotion switched ON
> ASPECTJ: aspectj.minimalModel=true: minimal model switched ON
> 
> If you are on windows, then you need to run eclipsec.exe.  Also, using the
> -consoleLog argument is good regardless of your platform (add to eclipse.ini as
> well).

(In reply to comment #23)
> I do not see these debug prints in the console.
> I am using:
> Eclipse AspectJ Development Tools
> 
> Version: 2.1.1.e36x-20100817-1900
> AspectJ version: 1.6.10.20100817163700
> 
> this is my eclipse.ini:
> -startup
> plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
> --launcher.library
> plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.0.v20100503
> -product
> org.eclipse.epp.package.java.product
> --launcher.defaultAction
> openFile
> --launcher.XXMaxPermSize
> 256M
> -showsplash
> org.eclipse.platform
> --launcher.XXMaxPermSize
> 256m
> --launcher.defaultAction
> openFile
> -vmargs
> -Dosgi.requiredJavaVersion=1.5
> -Xms40m
> -Xmx1024m
> -Daspectj.minimalModel=true
> -Daspectj.typeDemotion=true
> -consoleLog
> 
> (In reply to comment #21)
> > Are you on the latest dev build?  After that, all you need to do is add the
> > options to the eclipse.ini.
> > 
> > You will know that this option is turned on if you see things like this in your
> > system console:
> > 
> > ASPECTJ: aspectj.typeDemotion=true: type demotion switched ON
> > ASPECTJ: aspectj.minimalModel=true: minimal model switched ON
> > 
> > If you are on windows, then you need to run eclipsec.exe.  Also, using the
> > -consoleLog argument is good regardless of your platform (add to eclipse.ini as
> > well).

(In reply to comment #23)
> I do not see these debug prints in the console.
> I am using:
> Eclipse AspectJ Development Tools
> 
> Version: 2.1.1.e36x-20100817-1900
> AspectJ version: 1.6.10.20100817163700
> 
> this is my eclipse.ini:
> -startup
> plugins/org.eclipse.equinox.launcher_1.1.0.v20100507.jar
> --launcher.library
> plugins/org.eclipse.equinox.launcher.win32.win32.x86_1.1.0.v20100503
> -product
> org.eclipse.epp.package.java.product
> --launcher.defaultAction
> openFile
> --launcher.XXMaxPermSize
> 256M
> -showsplash
> org.eclipse.platform
> --launcher.XXMaxPermSize
> 256m
> --launcher.defaultAction
> openFile
> -vmargs
> -Dosgi.requiredJavaVersion=1.5
> -Xms40m
> -Xmx1024m
> -Daspectj.minimalModel=true
> -Daspectj.typeDemotion=true
> -consoleLog
> 
> (In reply to comment #21)
> > Are you on the latest dev build?  After that, all you need to do is add the
> > options to the eclipse.ini.
> > 
> > You will know that this option is turned on if you see things like this in your
> > system console:
> > 
> > ASPECTJ: aspectj.typeDemotion=true: type demotion switched ON
> > ASPECTJ: aspectj.minimalModel=true: minimal model switched ON
> > 
> > If you are on windows, then you need to run eclipsec.exe.  Also, using the
> > -consoleLog argument is good regardless of your platform (add to eclipse.ini as
> > well).
Comment 25 Andrew Eisenberg CLA 2010-08-26 15:18:53 EDT
In the workspace that you sent me, the PaperTests project uses the project root as a source folder, but has a separate destination folder.  When I made the change, memory usage went down dramatically.

What I think is happening is that the aspectj build is copying over all of the jars to the bin folder and this is causing an unnecessary spike in memory.

I was never able to get close to an OOME, though.  But this may fix things for you.
Comment 26 yoav CLA 2010-08-26 16:06:22 EDT
You were right. I closed this project and now I can build the much larger project with no problems. Very strange - is this something fixable?. 
Thanks!
Comment 27 Andrew Eisenberg CLA 2010-08-26 16:11:05 EDT
> is this something fixable?

As I mentioned, the best workaround is to move your source files into a source folder instead of the project root.  But, there is definitely a memory issue with this when running as an AspectJ project.  It is something I can look into.  

I'll resolve this bug and open a new one with a better description of what the real problem is.
Comment 28 Andrew Eisenberg CLA 2010-08-26 16:20:35 EDT
Created Bug 323745
Comment 29 yoav CLA 2010-08-26 16:52:40 EDT
Is this the right bug #?
(In reply to comment #28)
> Created Bug 323745
Comment 30 Andrew Eisenberg CLA 2010-08-26 16:57:43 EDT
(In reply to comment #29)
> Is this the right bug #?
Nope.  :)

Bug 323754