Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [aspectj-users] problems with load time weavingand rmicommunication


Ron,

My point is that if we as framework providers cannot robustly determine when a class loader should or should not be used for weaving what hope has the user. Requiring them to use extensive diagnostics or trial an error to set options is unsatisfactory. The type of configuration you suggest is a short term solution for experts only.

The weaver state should not pin the class loader; if it does it's a bug. The dramatic difference in the graphs between the FINAL and the DEVELOPMENT builds would suggest we have fixed such a bug. The only official link between a class loader and it's weaver is in Aj and that's a WeakReference.

Matthew Webster
AOSD Project
Java Technology Centre, MP146
IBM Hursley Park, Winchester,  SO21 2JN, England
Telephone: +44 196 2816139 (external) 246139 (internal)
Email: Matthew Webster/UK/IBM @ IBMGB, matthew_webster@xxxxxxxxxx

http://w3.hursley.ibm.com/~websterm/

Please respond to aspectj-users@xxxxxxxxxxx

Sent by:        aspectj-users-bounces@xxxxxxxxxxx

To:        <aspectj-users@xxxxxxxxxxx>
cc:        
Subject:        RE: [aspectj-users] problems with load time        weavingand        rmicommunication


Hi Matthew,
 
Let me look into the situation a little more. It appears that in Thilo’s case some other lightweight ClassLoaders are being allocated. My concern is that we aren’t going to be able to hard-code all the possibilities (e.g., if WebLogic or WebSphere or … have their own per object loader for RMI calls or for that matter dynamically generated proxies). In a case like that, at least there would be a possibility to improve memory use by configuration. It, to me, is very much like excluding by type name, or indeed by putting an aop.xml file in a place that’s visible to a loader, except that there are often loaders for which an aop.xml cannot be deployed in advance.
 
I do agree that the weaver state should be eligible for GC if the ClassLoader is. However, adding even 1 MB of Weaver State per remote proxy (which pins its ClassLoader) is a pretty significant amount of overhead!
 




From: aspectj-users-bounces@xxxxxxxxxxx [mailto:aspectj-users-bounces@xxxxxxxxxxx] On Behalf Of Matthew Webster
Sent:
Friday, March 10, 2006 8:19 AM
To:
aspectj-users@xxxxxxxxxxx
Subject:
RE: [aspectj-users] problems with load time weavingand rmicommunication

 

Ron,


The fix to https://bugs.eclipse.org/bugs/show_bug.cgi?id=120473 ensured that delegating class loaders were not used for weaving. If the class loaders in question fall into this category then they should also be excluded; not by name but using some other robust mechanism. Suggesting configuration implies a user may wish to weave on some occasions but not others. How would they make this decision? How would they know the name of the class loader to supply to the configuration? If we can't answer these questions then providing such an option will just confuse the user.


It would seem that these class loaders are short lived; if not the JVM would go OutOfMemory eventually without the help of LTW. Looking at the graphs it would seem that the first solution to this problem. releasing any references to the class loaders so that they and the associated weaver state can be collected, has been integrated. Reducing the overall footprint of the weaver continues and will also benefit this problem.


Matthew Webster
AOSD Project
Java Technology Centre, MP146
IBM Hursley Park, Winchester,  SO21 2JN, England
Telephone: +44 196 2816139 (external) 246139 (internal)
Email: Matthew Webster/UK/IBM @ IBMGB, matthew_webster@xxxxxxxxxx

http://w3.hursley.ibm.com/~websterm/

Please respond to aspectj-users@xxxxxxxxxxx

Sent by:        aspectj-users-bounces@xxxxxxxxxxx

To:        <aspectj-users@xxxxxxxxxxx>
cc:        
Subject:        
RE: [aspectj-users] problems with load time weaving and        rmicommunication


Hi Thilo,

I was suggesting that this might be a good change to make to AspectJ to
support cases like not creating weaving state when using RMI. However, as
you found, it isn't implemented in AspectJ (yet). It would be relatively
easy to do and I think it'd be interested to try it out.

Does <exclude loader="..."> seem like a good suggestion to you and others?
If so, we could put a bugzilla entry in to track it.

-----Original Message-----
From: aspectj-users-bounces@xxxxxxxxxxx
[mailto:aspectj-users-bounces@xxxxxxxxxxx] On Behalf Of Thilo Focke
Sent: Friday, March 10, 2006 3:19 AM
To: aspectj-users@xxxxxxxxxxx
Subject: Re: [aspectj-users] problems with load time weaving and
rmicommunication

Hi Rod,

thanks for your detailed message. Now i understand whats the problem
between load time weaving and rmi communication i think. According to
your suggestion i added the following to my aop.xml

<aspectj>
<weaver options="-verbose -proceedOnError -noWarn
-Xlintfile:META-INF/Xlint.properties">
...
<exclude loader="sun.reflect.DelegatingClassLoader"/>
<exclude loader="weblogic.utils.classloaders.GenericClassLoader"/>
</weaver>
...
</aspectj>

Unfortunately it has no effect on reducing the rmi-overhead. The console
still does the following output on each request:

info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26,
2006 at 03:39:19 GMT
info register classloader
weblogic.utils.classloaders.GenericClassLoader@9754965
info using
file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!/
META-INF/aop.xml
info register aspect de.myApp.PerformanceMonitor
info weaving 'de/myApp/service/bean/MyService_ee9ogc_HomeImpl_814_WLStub'
info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26,
2006 at 03:39:19 GMT
info register classloader
weblogic.utils.classloaders.GenericClassLoader@15456211
info using
file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!/
META-INF/aop.xml
info register aspect de.myApp.PerformanceMonitor
info weaving 'de/myApp/service/bean/MyService_ee9ogc_EOImpl_814_WLStub'
10.03.2006 08:48:48 de.ewetel.test.DummyServlet doThis_aroundBody4
INFO: Hello World!
10.03.2006 08:48:48 de.ewetel.test.DummyServlet doThis_aroundBody4

If you want to replicate my results: its just a simple servlet that
receives a user request. The doGet()-implementation of the servlet just
forwards the request to a method (doThis()) that opens a connection to a
remote EJB object (located in Bea Weblogic) and calls a
HelloWorld-method ;) On each request you get the output i pasted before.

if you have any more ideas how to reduce the overhead or need someone to
test a new overhead-reducing aspectj version please let me know ;)

Greetins,
Thilo



Ron Bodkin wrote:

>Hi Thilo,
>
>I think the results are encouraging in the sense that my optimizations have
>a big positive impact (1/6 the overhead with RMI, and at most 1/3 without
>RMI). As Andy and I noted, we're working to integrate them into AspectJ
>HEAD, in addition to the optimizations that Andy and Adrian have been
>working on.
>
>While your tests showed while the overhead is significantly reduced, there
>is still some noticeable overhead per ClassLoader. I'd like to optimize it
>further to allow nearly all the state in a case like this to be shared.
>
>Java VM's create ClassLoaders for RMI calls and proxies, presumably to
>isolate them so that "normal" classes don't see them. In this and in other
>circumstances, Java libraries often assume creating a ClassLoader is a
>lightweight operation... So I think AspectJ load-time weaving needs to keep
>focusing on adding small overhead in these cases.
>
>I do think there's an interesting short-term answer to scenarios like this.
>It would be helpful if the aop.xml file allowed excluding weaving in child
>ClassLoaders by name pattern, e.g.,
><aspectj>
>  <weaver>
>   <exclude loader="sun.reflect.DelegatingClassLoader"/>
>  </weaver>
>...
></aspectj>
>
>While this isn't ideal, it would make a huge difference in the runtime
>performance of load-time weaving (as in cases like this where you just
don't
>need to weave into the generated RMI classes).
>
>I'd be interested in trying to replicate your results in a simple example.
>Am I right that you are just opening a connection to a remote EJB object
>from inside your Web application?
>
>Integrating the optimizations for shared state is still pending some
>foundational work that's in progress...
>
>One thing that should help a lot is making sure that there are no hard
>references to the weaving state: conceptually whatever memory the weaver is
>allocating for this ClassLoader should be eligible for GC once the RMI
proxy
>object is eligible for GC... so some of the recent work in AspectJ CVS HEAD
>might just fix your memory problem. Of course, creating a megabyte of
>weaving state each time you use an RMI proxy is probably going to have some
>negative performance impact...
>
>-----Original Message-----
>From: aspectj-users-bounces@xxxxxxxxxxx
>[mailto:aspectj-users-bounces@xxxxxxxxxxx] On Behalf Of Thilo Focke
>Sent: Monday, March 06, 2006 1:44 AM
>To: aspectj-users@xxxxxxxxxxx
>Subject: Re: [aspectj-users] problems with load time weaving and rmi
>communication
>
>Hi guys,
>
>first of all thank you for your answers.
>i did some testing with the aspectjweavers from aspectj final, from the
>current development build and from Rods expermintal build from glassbox
>inspector. The results are alarmingly different.
>
>i did a little stress tests, that did about 3000 requests in a loop.
without
>rmi rmi communication there is no significant memory usage. with rmi
>communication and rods development aspectjweaver i was able to send nearly
>3000 requests. the memory usage went up to 500 MByte. with the
aspectjweaver
>from the aspectj final build and from the actual development build i was
>only able to send nearly 500 requests. the memory usage went up to 500MByte
>and i got an out of memory error. rods experimental build made a
significant
>difference in memory usage.
>
>i summarized my results on the following url:
>http://www.webengineer.de/ltw.html
>there you can find some screenhots from jconsole i made during my tests.
>
>i also switched on the -verbose option in my aop.xml. as matthew mentioned
>before, when using rmi communication there are created a lot of class
>loaders. see the following i pasted from my tomcat console:
>
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register classloader
>weblogic.utils.classloaders.GenericClassLoader@18880287
>info using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info register aspect de.pmon.PerformanceMonitor
>info weaving 'myTest/bean/Service_ee9ogc_HomeImpl_814_WLStub'
>info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register classloader
>weblogic.utils.classloaders.GenericClassLoader@32211177
>info using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info register aspect de.pmon.PerformanceMonitor
>info weaving 'myTest/bean/Service_ee9ogc_EOImpl_814_WLStub'
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register classloader
>weblogic.utils.classloaders.GenericClassLoader@20872473
>info using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info register aspect de.pmon.PerformanceMonitor
>info weaving 'myTest/bean/Service_ee9ogc_HomeImpl_814_WLStub'
>info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register classloader
>weblogic.utils.classloaders.GenericClassLoader@22373430
>info using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info register aspect de.pmon.PerformanceMonitor
>info weaving 'myTest/bean/Service_ee9ogc_EOImpl_814_WLStub'
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register classloader
>weblogic.utils.classloaders.GenericClassLoader@25411170
>info using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info register aspect de.pmon.PerformanceMonitor
>info weaving 'myTest/bean/Service_ee9ogc_HomeImpl_814_WLStub'
>info AspectJ Weaver Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register classloader
>weblogic.utils.classloaders.GenericClassLoader@10284000
>info using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info register aspect de.pmon.PerformanceMonitor
>info weaving 'myTest/bean/Service_ee9ogc_EOImpl_814_WLStub'
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>06.03.2006 10:17:36 myTest.DummyServlet doThis_aroundBody4
>
>can you explain me why it is necessary to register a new classloader on
each
>rmi request?
>we planned to integrate the permormance monitoring in one of our production
>systems. but as long as the memory increases on each rmi request we think
it
>is not a good idea to integrate it. currently we only use it on our test
>systems (unfortunatley).
>
>greetings,
>thilo
>
>
>
>
>Ron Bodkin wrote:
>
>  
>
>>Thilo,
>>
>>Just as a quick follow up, the AspectJ 1.5.0 release adds some nontrivial
>>overhead per ClassLoader where you do weaving. When you are using RMI, the
>>VM can create a lot of different ClassLoaders for reflective acessors.
>>
>>The experimental LTW version I pointed you at allows for sharing aspect
>>definitions and AspectJ delegates between ClassLoaders where it is safe
(it
>>tests for nondelegation), so it should make a significant difference. I
>>haven't yet created a patch for this optimization, since as Andy noted,
I'm
>>working with him to integrate some foundational memory improvements,
>>bringing them up to generally usable production quality :-)
>>
>>The same issue applies for JSP's, where many containers create a large
>>number of lightweight ClassLoaders and maintaining heavyweight weaving
>>    
>>
>state
>  
>
>>for each results in major overhead.
>>
>>-----Original Message-----
>>From: aspectj-users-bounces@xxxxxxxxxxx
>>[mailto:aspectj-users-bounces@xxxxxxxxxxx] On Behalf Of Thilo Focke
>>Sent: Friday, March 03, 2006 8:06 AM
>>To: aspectj-users@xxxxxxxxxxx
>>Subject: Re: [aspectj-users] problems with load time weaving and rmi
>>communication
>>
>>Hi Matthew and Andy,
>>
>>1. The memory usage when using RMI without the aspect ist absolutely
>>constant and in acceptable ranges (about 30-40 MByte). The memory usage
>>increases permanently when using the aspect. When using the aspect on code
>>without RMI communication the memory usage is constant (about 100MByte).
>>
>>2. IŽm using  AspectJ 1.5.0, released on 20th December, 2005
>>
>>3. My aop.xml looks like this
>>
>><!DOCTYPE aspectj PUBLIC "-//AspectJ//DTD//EN"
>>"http://www.aspectj.org/dtd/aspectj_1_5_0.dtd">
>><aspectj>
>>   <weaver options="-proceedOnError -noWarn
>>-Xlintfile:META-INF/Xlint.properties">
>>       <include within="my.package..*"/>      
>>   </weaver>
>>   <aspects>        
>>       <aspect name="MyMonitor"/>
>>   </aspects>
>></aspectj>
>>
>>Thanks a lot,
>>Thilo
>>
>>
>>
>>
>>--------------->
>>
>>Thilo,
>>
>>Can I ask some questions:
>>1. What is the memory usage when you use RMI but not the aspect?
>>2. Which version of AspectJ are you using? We have made some enhancements
>>recently in memory usage.
>>3. Do you use -verbose in the weaver options of aop.xml? If so could you
>>post it. I think the problem might to do with creating lots of class
>>loaders, each with a weaver and the accompanying footprint.
>>
>>Cheers
>>
>>Matthew Webster
>>AOSD Project
>>Java Technology Centre, MP146
>>IBM Hursley Park, Winchester,  SO21 2JN, England
>>Telephone: +44 196 2816139 (external) 246139 (internal)
>>Email: Matthew Webster/UK/IBM @ IBMGB, matthew_webster@xxxxxxxxxx
>>_______________________________________________
>>aspectj-users mailing list
>>aspectj-users@xxxxxxxxxxx
>>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>>
>>
>>_______________________________________________
>>aspectj-users mailing list
>>aspectj-users@xxxxxxxxxxx
>>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>>
>>    
>>
>
>_______________________________________________
>aspectj-users mailing list
>aspectj-users@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>
>_______________________________________________
>aspectj-users mailing list
>aspectj-users@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>
>  
>

--
EWE TEL GmbH
Thilo Focke
IP/IT-Systeme & Netze
Cloppenburger Strasse 310, D-26133 Oldenburg
Telefon: +49 (0)441 8000-2863
Telefax: +49 (0)441 8000-2799
thilo.focke@xxxxxxxxx
http://www.ewetel.de

_______________________________________________
aspectj-users mailing list
aspectj-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/aspectj-users

_______________________________________________
aspectj-users mailing list
aspectj-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/aspectj-users
_______________________________________________
aspectj-users mailing list
aspectj-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/aspectj-users


Back to the top