Re: [aspectj-users] Heap space problem

[Andy Clement <andrew.clement@xxxxxxxxx>]

Hi,

Best thing to do would be to share the data with me if you can, and
tell me the offending handle.

send it to my gmail ID if you don't feel like attaching it to the
bugzilla I raised.

If we can determine if there is something unusual about the type that
did not get removed from the model, we can enhance the discard logic.

Andy

On 27 January 2011 14:12, Florian Weiss <florian.weiss@xxxxxxxxxx> wrote:
> Ok
>
> I definitely have a handle in the hierarchy that is not listed in the relationship map.
>
> What I did: I copied a Handle of a java file searched for it and it only appeared in the relationship map.
> Hope that proves that it is not necessary.
>
> How can i figure out why it is there?
>
> Florian
>
> -----Original Message-----
> From: aspectj-users-bounces@xxxxxxxxxxx [mailto:aspectj-users-bounces@xxxxxxxxxxx] On Behalf Of Andy Clement
> Sent: Donnerstag, 27. Januar 2011 23:01
> To: aspectj-users@xxxxxxxxxxx
> Subject: Re: [aspectj-users] Heap space problem
>
>> minimalModel=true: The hierarchy and the relationship map have 6400 lines
>> no option: 22800 lines
>
> nice.
>
> The output 'dump' includes two things:
> 1. a hierarchy of the types being held onto
> 2. a set of relationships between elements of those types.
>
> Here is a tiny example for my little project:
>
> 13:54:35 Printing crosscutting model for A2
> 13:54:35 Hierarchy:
> =A2/src
>  =A2/src<com.bar
>    =A2/src<com.bar*X.aj
>      =A2/src<com.bar*X.aj%com.bar
>      =A2/src<com.bar*X.aj#
>      =A2/src<com.bar*X.aj'X
>        =A2/src<com.bar*X.aj'X&before
>  =A2/src<com.foo
>    =A2/src<com.foo{C.java
>      =A2/src<com.foo{C.java%com.foo
>      =A2/src<com.foo{C.java#
>      =A2/src<com.foo{C.java[C
>        =A2/src<com.foo{C.java[C~m
>
> Relationship map:
> =A2/src<com.bar*X.aj'X&before ::
>        =A2/src<com.bar*X.aj'X&before --advises--> [=A2/src<com.foo{C.java[C~m]
> =A2/src<com.foo{C.java[C~m ::
>        =A2/src<com.foo{C.java[C~m --advised by--> [=A2/src<com.bar*X.aj'X&before]
>
> The cryptic strings are 'handles' in a JDT compatible format, they
> uniquely identify a component of a type.  The hierarchy here includes
> 2 types, X and C.  So I can see my aspect X.aj (=A2/src<com.bar*X.aj).
>  I can also see my target C.java (=A2/src<com.foo{C.java).
>
> The relationship map then lists relationships between handles.  I can
> see the before advice in my X.aj advises the method m in my C.java
> (relationships are in pairs so I also see the inverse 'advised by'.
>
> The basic rule of a minimal model is that the only things are
> persisted are aspects (always) or types targeted by a relationship
> from an aspect.  So above as C.java is targeted by a relationship, it
> is held onto.  My project also has a file D.java in it (alongside C)
> but because no aspect affected it, it is not in the model above.
>
> So, basically find something you think shouldn't be there, then take a
> look in the "relationship map" section - search for the handle of the
> file/type you dont think should be there.  If it really isn't
> mentioned we need to figure out why it was held onto.
>
> Andy
>
> On 27 January 2011 13:51, Florian Weiss <florian.weiss@xxxxxxxxxx> wrote:
>> Hi
>>
>> Just did another run with just Project1.
>>
>>
>> So there is an improvement. But still if I check some of the mentioned files in the dump of the smaller hierarchy. They have no markers or something. How do I figure out if a file should be mentioned or not?
>>
>> Florian
>>
>> -----Original Message-----
>> From: aspectj-users-bounces@xxxxxxxxxxx [mailto:aspectj-users-bounces@xxxxxxxxxxx] On Behalf Of Andy Clement
>> Sent: Donnerstag, 27. Januar 2011 22:17
>> To: aspectj-users@xxxxxxxxxxx
>> Subject: Re: [aspectj-users] Heap space problem
>>
>> Hi,
>>
>> I just had a look and the system properties (can be easier than
>> managing eclipse settings) are:
>>
>> aspectj.typeDemotion=true
>> aspectj.minimalModel=true
>>
>> but it sounds like you are getting them through as regular project
>> settings anyway.
>>
>>> "ClassFileChangeChecking: no reported changes in that state". Somtimes it is alot of Messages i get eg. when soemthing changed
>>
>> I can't immediately see how those messages related to the two flag
>> settings.  Let's ignore them for now.
>>
>>> The only difference that i see is the time when eclipse crashes with settings off it crashes earlier.
>>
>> That sounds like what I might expect - it takes longer to run out of memory.
>>
>>> What do you mean by alot of files? I actually don't know the exact number of hits but i would guess somewhere around 1000. Can't figure it out exactly since the build doesn't finish.
>>
>> I was more thinking as a percentage.  If you are hitting 99% of the
>> files from the aspects then 'minimalModel' won't do much since it only
>> discards model entries for types that are not hit by aspects.  Your
>> 1000 doesn't sound a lot given the size of your project, so I'd hope
>> that setting would at least do something.  We can try something, but
>> I'm not sure if it has been tried with such large projects before.  On
>> the AJDT event trace view is a button to dump the structure models for
>> all the projects.  It is the two boxes with arrows going between them.
>>  If you, for example, closed all but 1 or two of the base projects,
>> you could click that dump button after builds with and without using
>> the minimalModel setting.  In the latter case it should be a lot
>> smaller - if we can observe rogue entries in there (types not affected
>> by aspects that haven't been discarded), I can further improve the
>> minimal model code.
>>
>> I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=335619 to track
>> this issue for you (cc yourself on it) - maybe we can exchange
>> findings on there, or if you want to email me pieces of the output
>> (the model) we could do that.
>>
>>> What i was wondering, now i have alwas two dependencies between projects, one of type src and one of type lib (the bin folder of the same project). Does this lead to any problems?
>>> Example:
>>>
>>> Project2: .classpath
>>> ...
>>> <classpathentry exported="true" kind="src" path="/Project1"/>
>>> <classpathentry exported="true" kind="lib" path="/Project1/bin/">
>>>        <attributes>
>>>                <attribute name="org.eclipse.ajdt.aspectpath" value="org.eclipse.ajdt.aspectpath"/>
>>>        </attributes>
>>> </classpathentry>
>>
>> That should be fine.  If you were using inpath I'd be more concerned.
>>
>>> Just tried to build it again with -Xset:minimalModel=true,typeDemotion=true set in the global compiler preferences.
>>> And eclipse started with -consolelog.
>>> On the console i get: "typeDemotion=true: type demotion switched ON" per project that is being build.
>>
>> that sounds right, there isn't a message for minimalModel becoming
>> true right now.  We can actually see into what type demotion is doing
>> if you also turn on typeDemotionDebug=true - I believe that output
>> will go to the consolelog.  Type demotion (unlike minimal model) is a
>> bit of a memory<>performance trade off - activating it causes stuff to
>> be more eagerly discarded, hoping that it isn't required later, if it
>> is required then there is a cost to pull it back into memory.  Minimal
>> Model should just be a straight improvement in memory consumption.
>>
>>>
>>> Project1:
>>> 21:1:22 Types affected during build = 899
>>> 21:1:23 Timer event: 33609ms: Total time spent in AJBuilder.build()
>>> 21:1:24 Created 71 markers in 899 files
>>>
>>> Project2:
>>> 21:3:19 Types affected during build = 1926
>>> 21:3:20 Timer event: 116984ms: Total time spent in AJBuilder.build()
>>> 21:3:22 Created 198 markers in 1926 files
>>>
>>> Project3:
>>> 21:17:7 Types affected during build = 4441
>>> 21:17:9 Timer event: 828563ms: Total time spent in AJBuilder.build()
>>> 21:17:10 Created 3 markers in 4441 files
>>
>> That Project3 is taking a long time, but I suppose it is 4k files.  Do
>> you know if the VM is already thrashing at this point?  You should be
>> able to see that if you either attach JConsole or if you turn on the
>> heap info in eclipse (Eclipse>Preferences>General>Show Heap Status -
>> it will turn up in the lower right of eclipse).
>>
>> From what you describe, I am suspicious of minimal model not behaving
>> as expected (not being aggressive enough perhaps).  If you are up for
>> it we could try some of what I suggested above.  Close all but a few
>> projects, turn the flags all OFF.  Do a build, press that model
>> dumping button, see what we get.  Turn on minimalModel, do a build,
>> dump again, hopefully we should not see files in there that are not
>> affected by aspects (You could even just do the second part of this
>> and dump the model with the flag off - have a look for anything that
>> shouldn't be there).
>>
>> cheers
>> Andy
>> _______________________________________________
>> aspectj-users mailing list
>> aspectj-users@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/aspectj-users
>> _______________________________________________
>> aspectj-users mailing list
>> aspectj-users@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/aspectj-users
>>
> _______________________________________________
> aspectj-users mailing list
> aspectj-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/aspectj-users
> _______________________________________________
> aspectj-users mailing list
> aspectj-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/aspectj-users
>