Comment 1 Andrew Clement 2009-03-10 11:39:31 EDT

So the initial build of the project is fine?

Can I ask a bit more about the specifics:
- how much source is there to compile? 100s of classes? 1000s? 10000s?
- how many aspects? are they highly pervasive?
- are you using input jars for weaving?
- are there aspects coming in on the aspectpath?

But without the ability to recreate this locally, its going to be tough to address.  I am not aware of any other users with a memory leak problem.  Is there any way you can share the failing project with me? (not necessarily through a public channel like bugzilla)

Comment 2 Stefan Schedl 2009-03-10 12:12:18 EDT

(In reply to comment #1)

The initial build work, but I think this is because there is enough free memory, that could be wasted. 
I have 9 Aspectj projects with summary 12 aspects which weaves into 9 big classes. one projects provides abstract aspects which will be implemented by the other projects. the project with the abstract aspect is on the aspectpath of the other projects. 
The failing project is from an client of me, so i could not share ist. But i can try to reproduce it in an test-project. 

Comment 3 Andrew Eisenberg 2009-03-13 20:20:56 EDT

Please attach a test project and I will look into this.

From what you've seen so far, are the OOME fairly consistent or are they happening all over the place?

Comment 4 Andrew Clement 2009-03-13 21:01:55 EDT

i dont think this will be an AJDT problem, it is more likely an AJ problem.

Comment 5 Stefan Schedl 2009-03-16 10:40:26 EDT

It is much more difficult to create a test project than I thought...
Is there an other possibility to help you finding the bug?

The OOME are fairly consistent. It's impossible to work with our aspectj projects. as an workaround we disabled the AspectJ-Builder and write an ant-task, that does the weaving. but this is a quite ugly solution.


(In reply to comment #3)
> Please attach a test project and I will look into this.
> 
> From what you've seen so far, are the OOME fairly consistent or are they
> happening all over the place?
> 

Comment 6 Andrew Clement 2009-03-16 18:29:38 EDT

I wonder if it is related to the use of an aspect library as we don't do much memory usage testing of that, we typically just run with multiple projects that are inter-dependent.  

You say:
> one projects provides abstract aspects which will be implemented by
> the other projects. the project with the abstract aspect is on the aspectpath
> of the other projects. 

Do you package the output of one of those projects into a jar that is on the aspectpath of the other?  or do you just use 'regular' bin folder project dependencies?  From the initial stack trace:

org.aspectj.weaver.bcel.BcelWeaver.addAspectsFromJarFile(BcelWeaver.java:259)
        at
org.aspectj.weaver.bcel.BcelWeaver.addLibraryJarFile(BcelWeaver.java:236)

it looks like you are passing jars on the aspectpath - I am just wondering if it will behave any better from an incremental point of view if you use directories.

Comment 7 Stefan Schedl 2009-03-25 11:27:41 EDT

Wie use regular bin folder. the structure of our aspectj-project is:

aj-project 1: with the abstract aspects

aj-project 2: concret implementation of the aspects
  - aspectj-build/aspect-path/add-project: aj-project 1

aj-project 3-9: like aj-project 2

(each project has many non-aspectj-project/library dependencies)


(In reply to comment #6)
> I wonder if it is related to the use of an aspect library as we don't do much
> memory usage testing of that, we typically just run with multiple projects that
> are inter-dependent.  
> 
> You say:
> > one projects provides abstract aspects which will be implemented by
> > the other projects. the project with the abstract aspect is on the aspectpath
> > of the other projects. 
> 
> Do you package the output of one of those projects into a jar that is on the
> aspectpath of the other?  or do you just use 'regular' bin folder project
> dependencies?  From the initial stack trace:
> 
> org.aspectj.weaver.bcel.BcelWeaver.addAspectsFromJarFile(BcelWeaver.java:259)
>         at
> org.aspectj.weaver.bcel.BcelWeaver.addLibraryJarFile(BcelWeaver.java:236)
> 
> it looks like you are passing jars on the aspectpath - I am just wondering if
> it will behave any better from an incremental point of view if you use
> directories.
> 

Comment 8 Andrew Clement 2009-03-25 16:05:36 EDT

I'm going to try simulating this in my setup with 16 aspectj projects in a dependency hierarchy.  I'll create a new one with an abstract aspect in then extend it in a few of the downstream ones.

Comment 9 Andrew Clement 2009-03-25 16:39:39 EDT

Using yourkit.  Some baseline information.  On startup of the workspace, with nothing built and at the moment multiple aspectj projects but NO aspects (900M Xmx)

heap: 39M (used) non-heap: 41M (used) loaded classes: 6771
clean and full build: heap: 270 (after force GC 163M) nonheap: 69 classes:10199
clean and full build: heap: 351M (after force GC 181M) nonheap: 72M classes:10373
clean and full build: heap: 308M (after force GC 254M) nonheap: 72M classes: 10379
clean and full build: heap: 320M (after force GC 256M) nonheap: 72M classes: 10379

Interesting, but now I'll introduce a trace aspect via aspectpath to a few projects.

Aspect is in a Tracer project:

public abstract aspect BaseTrace {
abstract pointcut interesting();
	before(): interesting() {}
	after(): interesting() {}
}

added to aspectpath of weaver/util/org.aspectj.matcher

Created same subaspect in each project.

public aspect Tracer extends BaseTrace {
 pointcut interesting(): execution(* *(..)) && !within(BaseTrace+);
}


Now clean start of the workbench:

heap: 293M (force gc: 221M)
clean/full build. 
heap: 398M (force gc: 258M)

new public method in util/FileUtil (affected by the tracer aspect)
(build time here was a bit ugly... wonder why)
heap: 453M (force gc: 248M)
another new public method in util/FileUtil 
heap: 396M (force gc: 250M)

No terrible leak that I can observe - but I am only using simple before/after advice.  Can you tell me what kind of language constructs you are using in your aspects?

I'm going to take a quick look into the build time for this scenario and why adding that public method was so expensive for the incremental build.

Comment 10 Andrew Clement 2009-03-25 17:45:29 EDT

Created bug 270033 to look into the observed problem with incremental building when there are incoming aspects

Comment 11 Stefan Schedl 2009-03-30 05:07:08 EDT

We are using 
pointcut foo: execution(* package.*(..));
pointcut foo(): call(* class.method*(..));
abstract pointcut foo(); 
         before():
	after(): 
	after() throwing():
	

Comment 12 Andrew Clement 2009-03-30 13:24:53 EDT

Although I have not fixed any memory issues, i have made incremental builds of aspectpath configurations a bit more optimal - that is in recent dev builds of AJDT, so you might want to upgrade.  (it was bug 270033)

Fixing the memory issue here is hard as I just cannot recreate any kind of leak.  Yes, the Bcel World objects consuming a large chunk of memory (as shown in the analyzer report you attached) but that is not a surprise to me.  Given that they only store type data I am not sure how they can leak if it is the same types across incremental builds.  This doesn't mean there is no leak, I just can't debug it if I can't recreate it :(

Some of the type references in the world are held via weak/soft references - depending on the VM you are using these can behave in a peculiar way.  Depending on your VM you can get the references to be collected early using a -X option, see something like that mentioned here:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=227484#c31

My plan is to shrink these world objects more proactively in 1.6.5 because I can't rely on Weak/Soft references and this will reduce the memory required but if they are leaking it will just delay the failure you are seeing.

Comment 13 Stefan Schedl 2009-06-05 11:41:09 EDT

I tested it with the actual 3.4 dev build. without the options descriped in bug #227484 my eclipse still crashes on every full-rebuild. But with this option it will no longer crash (the memory usage is still quite high)

> Some of the type references in the world are held via weak/soft references -
> depending on the VM you are using these can behave in a peculiar way. 
> Depending on your VM you can get the references to be collected early using a
> -X option, see something like that mentioned here:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=227484#c31
this helps nothing

With the new aspectj version i get another error, any idea?

java.lang.IllegalArgumentException: Path must include project and resource name: /org.apache.xerces
	at org.eclipse.core.runtime.Assert.isLegal(Assert.java:64)
	at org.eclipse.core.internal.resources.Workspace.newResource(Workspace.java:1628)
	at org.eclipse.core.internal.resources.Container.getFolder(Container.java:137)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.pathToFile(CoreOutputLocationManager.java:630)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:591)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.mapProject(CoreOutputLocationManager.java:573)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:601)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:586)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.mapProject(CoreOutputLocationManager.java:573)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:601)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.mapProject(CoreOutputLocationManager.java:573)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:601)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.mapProject(CoreOutputLocationManager.java:573)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.initDeclaringProjectsMap(CoreOutputLocationManager.java:561)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.findDeclaringProject(CoreOutputLocationManager.java:533)
	at org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.discoverChangesSince(CoreOutputLocationManager.java:653)
	at org.aspectj.ajde.core.internal.OutputLocationAdapter.discoverChangesSince(OutputLocationAdapter.java:62)
	at org.aspectj.ajdt.internal.core.builder.AjState.classFileChangedInDirSinceLastBuildRequiringFullBuild(AjState.java:460)
	at org.aspectj.ajdt.internal.core.builder.AjState.classpathChangedAndNeedsFullBuild(AjState.java:1009)
	at org.aspectj.ajdt.internal.core.builder.AjState.pathChange(AjState.java:822)
	at org.aspectj.ajdt.internal.core.builder.AjState.prepareForNextBuild(AjState.java:255)
	at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performBuild(AjBuildManager.java:211)
	at org.aspectj.ajdt.internal.core.builder.AjBuildManager.incrementalBuild(AjBuildManager.java:185)
	at org.aspectj.ajde.core.internal.AjdeCoreBuildManager.performBuild(AjdeCoreBuildManager.java:127)
	at org.aspectj.ajde.core.AjCompiler.build(AjCompiler.java:88)
	at org.eclipse.ajdt.core.builder.AJBuilder.build(AJBuilder.java:245)
	at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:633)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:170)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:201)
	at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:253)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
	at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:256)
	at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:309)
	at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:341)
	at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:140)
	at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:238)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

By the way, thanks for the good support!

Comment 14 Andrew Eisenberg 2009-06-05 16:53:51 EDT

This looks like an AJDT problem.  I think there is some entry in your .classpath file that is not being properly handled by AJDT.  I have a pretty good idea what might be going on, but I need some more information.  

I would like to see the .classpath of the project that references org.apache.xerces (if there is proprietary information in it, then you can rename the proprietary pieces to something else).

(In reply to comment #13)
> With the new aspectj version i get another error, any idea?
> 
> java.lang.IllegalArgumentException: Path must include project and resource
> name: /org.apache.xerces
>         at org.eclipse.core.runtime.Assert.isLegal(Assert.java:64)
>         at
> org.eclipse.core.internal.resources.Workspace.newResource(Workspace.java:1628)
>         at
> org.eclipse.core.internal.resources.Container.getFolder(Container.java:137)
>         at
> org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.pathToFile(CoreOutputLocationManager.java:630)
>         at
> org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:591)
>         at
> org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.mapProject(CoreOutputLocationManager.java:573)
>         at
> org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:601)
>         at
> org.eclipse.ajdt.internal.core.ajde.CoreOutputLocationManager.handleClassPathEntry(CoreOutputLocationManager.java:586)
>         at
> 

Comment 15 Stefan Schedl 2009-06-07 16:09:01 EDT

xerces is a plug-in dependency. the xerces plugin is a updated copy of the org.apache.xerces plugin, bundled with eclipse, located in my workspace
form the project, that contains the xerces reference (project1): 
the .classpath:
<?xml version="1.0" encoding="UTF-8"?>
<classpath>
	<classpathentry kind="src" path="src"/>
	<classpathentry kind="con" path="org.eclipse.pde.core.requiredPlugins"/>
	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/J2SE-1.5"/>
	<classpathentry kind="output" path="bin"/>
</classpath>

manifest.mf: 

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: 
Bundle-SymbolicName: com.project1; singleton:=true
Bundle-Version: 3.0.0
Bundle-ClassPath: .
Bundle-Vendor: 
Bundle-Localization: plugin
Require-Bundle: org.eclipse.core.runtime,
 org.eclipse.core.resources,
 org.eclipse.ui,
 org.eclipse.gef,
 org.eclipse.ui.views,
 org.eclipse.ui.ide,
 org.apache.xerces;bundle-version="[2.9.0,3.0.0)",
 com.sun.jai;bundle-version="1.3.0"
Bundle-ActivationPolicy: lazy
Bundle-RequiredExecutionEnvironment: J2SE-1.5

classpath from the aspectj project:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
	<classpathentry excluding="/server/aspectj/RuntimeTimeTracing.aj|/server/aspectj/RuntimeTracing.aj" kind="src" path="src"/>
	<classpathentry exported="true" kind="src" path="/project1"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.api"/>
	<classpathentry exported="true" kind="src" path="/com.server">
		<attributes>
			<attribute name="org.eclipse.ajdt.aspectpath" value="org.eclipse.ajdt.aspectpath"/>
		</attributes>
	</classpathentry>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.support"/>
	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.context"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.internationalisation.generator"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/org.apache.commons-logging.libs"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.lookup"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.support.webapplication"/>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.auth"/>
	<classpathentry exported="true" kind="con" path="org.eclipse.ajdt.core.ASPECTJRT_CONTAINER">
		<attributes>
			<attribute name="org.eclipse.jst.component.dependency" value=""/>
		</attributes>
	</classpathentry>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/com.eaio.uuid.libs"/>
	<classpathentry exported="true" kind="lib" path="lib/quartz-1.6.5.jar">
		<attributes>
			<attribute name="org.eclipse.jst.component.dependency" value="../"/>
		</attributes>
	</classpathentry>
	<classpathentry exported="true" kind="lib" path="lib/commons-lang-2.4.jar">
		<attributes>
			<attribute name="org.eclipse.jst.component.dependency" value="../"/>
		</attributes>
	</classpathentry>
	<classpathentry exported="true" kind="lib" path="lib/commons-collections-3.2.jar">
		<attributes>
			<attribute name="org.eclipse.jst.component.dependency" value="../"/>
		</attributes>
	</classpathentry>
	<classpathentry combineaccessrules="false" exported="true" kind="src" path="/org.hibernate.annotations"/>
	<classpathentry kind="output" path="bin"/>
</classpath>

Comment 16 Andrew Eisenberg 2009-06-08 13:30:02 EDT

Raised bug 279497 to deal with the issue described in c13.

Comment 17 Andrew Eisenberg 2009-09-30 14:37:26 EDT

Move to the 2.0.2 release.

Comment 18 Andrew Eisenberg 2010-04-28 19:20:25 EDT

No longer slated for the next release.