Subject: RE: [aspectj-users] problems with loadtime
weavingand rmicommunication
I think there are
valid cases where a user can know that they donât want to weave in a loader
_for their application_, for
just the same reasons that we have exclude clauses in an aop.xml file. This
isnât implying that no one would ever need to weave the loader, just that for
my aspects, I donât need to affect the types loaded by this
loader.
The graphs that Thilo
posted were a little hard to read in that the 1.5.0 vs. dev builds both showed
memory use after just 500 requests and for the experimental build that I made
showed memory use after 3000 requestsâ¦
I agree that the
weaver state shouldnât pin class loaders, but the memory that is added per
proxy can be significant. Iâd like to dig in a bit to see if I can reproduce
Thiloâs results to understand what loader(s) are in effect. Maybe a good first
step is to allow another trace option just to trace the construction of
weavers for a loader. It would be nice to have a means to get statistics on
the weaving container too (a JMX remote option
maybe)â¦
From:
aspectj-users-bounces@xxxxxxxxxxx [mailto:aspectj-users-bounces@xxxxxxxxxxx]
On Behalf Of Matthew
Webster
Sent: Friday, March
10, 2006 9:32 AM
To:
aspectj-users@xxxxxxxxxxx
Subject: RE: [aspectj-users] problems
with loadtime weavingand rmicommunication
Ron,
My point is that if we as
framework providers cannot robustly determine when a class loader should or
should not be used for weaving what hope has the user. Requiring them to use
extensive diagnostics or trial an error to set options is unsatisfactory. The
type of configuration you suggest is a short term solution for experts
only.
The weaver state should not
pin the class loader; if it does it's a bug. The dramatic difference in the
graphs between the FINAL and the DEVELOPMENT builds would suggest we have
fixed such a bug. The only official link between a class loader and it's
weaver is in Aj and that's a WeakReference.
Matthew Webster
AOSD
Project
Java Technology Centre, MP146
IBM Hursley
Park, Winchester, SO21
2JN, England
Telephone: +44 196
2816139 (external) 246139 (internal)
Email: Matthew Webster/UK/IBM @
IBMGB, matthew_webster@xxxxxxxxxx
http://w3.hursley.ibm.com/~websterm/
Please
respond to aspectj-users@xxxxxxxxxxx
Sent by:
aspectj-users-bounces@xxxxxxxxxxx
To:
<aspectj-users@xxxxxxxxxxx>
cc:
Subject:
RE: [aspectj-users] problems
with load time weavingand
rmicommunication
Hi
Matthew,
Let me look into the
situation a little more. It appears that in Thiloâs case some other
lightweight ClassLoaders are being allocated. My concern is that we arenât
going to be able to hard-code all the possibilities (e.g., if WebLogic or
WebSphere or ⦠have their own per object loader for RMI calls or for that
matter dynamically generated proxies). In a case like that, at least there
would be a possibility to improve memory use by configuration. It, to me, is
very much like excluding by type name, or indeed by putting an aop.xml file in
a place thatâs visible to a loader, except that there are often loaders for
which an aop.xml cannot be deployed in advance.
I do agree that the
weaver state should be eligible for GC if the ClassLoader is. However, adding
even 1 MB of Weaver State per remote proxy (which pins its
ClassLoader) is a pretty significant amount of overhead!
From:
aspectj-users-bounces@xxxxxxxxxxx [mailto:aspectj-users-bounces@xxxxxxxxxxx]
On Behalf Of Matthew
Webster
Sent: Friday, March
10, 2006 8:19 AM
To:
aspectj-users@xxxxxxxxxxx
Subject: RE: [aspectj-users] problems
with load time weavingand rmicommunication
Ron,
The fix to
https://bugs.eclipse.org/bugs/show_bug.cgi?id=120473 ensured that delegating
class loaders were not used for weaving. If the class loaders in question fall
into this category then they should also be excluded; not by name but using
some other robust mechanism. Suggesting configuration implies a user may wish
to weave on some occasions but not others. How would they make this decision?
How would they know the name of the class loader to supply to the
configuration? If we can't answer these questions then providing such an
option will just confuse the user.
It would
seem that these class loaders are short lived; if not the JVM would go
OutOfMemory eventually without the help of LTW. Looking at the graphs it would
seem that the first solution to this problem. releasing any references to the
class loaders so that they and the associated weaver state can be collected,
has been integrated. Reducing the overall footprint of the weaver continues
and will also benefit this problem.
Matthew
Webster
AOSD Project
Java Technology Centre, MP146
IBM Hursley
Park, Winchester, SO21
2JN, England
Telephone: +44 196
2816139 (external) 246139 (internal)
Email: Matthew Webster/UK/IBM @
IBMGB, matthew_webster@xxxxxxxxxx
http://w3.hursley.ibm.com/~websterm/
Please
respond to aspectj-users@xxxxxxxxxxx
Sent by:
aspectj-users-bounces@xxxxxxxxxxx
To:
<aspectj-users@xxxxxxxxxxx>
cc:
Subject:
RE: [aspectj-users] problems
with load time weaving and
rmicommunication
Hi
Thilo,
I was suggesting that this might be a good change to make to
AspectJ to
support cases like not creating weaving state when using RMI.
However, as
you found, it isn't implemented in AspectJ (yet). It would be
relatively
easy to do and I think it'd be interested to try it
out.
Does <exclude loader="..."> seem like a good suggestion to
you and others?
If so, we could put a bugzilla entry in to track
it.
-----Original Message-----
From:
aspectj-users-bounces@xxxxxxxxxxx
[mailto:aspectj-users-bounces@xxxxxxxxxxx]
On Behalf Of Thilo Focke
Sent: Friday, March 10, 2006 3:19 AM
To:
aspectj-users@xxxxxxxxxxx
Subject: Re:
[aspectj-users] problems with load time weaving
and
rmicommunication
Hi Rod,
thanks for your detailed
message. Now i understand whats the problem
between load time weaving and
rmi communication i think. According to
your suggestion i added the
following to my aop.xml
<aspectj>
<weaver options="-verbose
-proceedOnError -noWarn
-Xlintfile:META-INF/Xlint.properties">
...
<exclude
loader="sun.reflect.DelegatingClassLoader"/>
<exclude
loader="weblogic.utils.classloaders.GenericClassLoader"/>
</weaver>
...
</aspectj>
Unfortunately
it has no effect on reducing the rmi-overhead. The console
still does the
following output on each request:
info AspectJ Weaver Version
LTWExperimental built on Thursday Jan 26,
2006 at 03:39:19 GMT
info
register classloader
weblogic.utils.classloaders.GenericClassLoader@9754965
info using
file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!/
META-INF/aop.xml
info
register aspect de.myApp.PerformanceMonitor
info weaving
'de/myApp/service/bean/MyService_ee9ogc_HomeImpl_814_WLStub'
info AspectJ
Weaver Version LTWExperimental built on Thursday Jan 26,
2006 at 03:39:19
GMT
info register classloader
weblogic.utils.classloaders.GenericClassLoader@15456211
info using
file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!/
META-INF/aop.xml
info
register aspect de.myApp.PerformanceMonitor
info weaving
'de/myApp/service/bean/MyService_ee9ogc_EOImpl_814_WLStub'
10.03.2006
08:48:48 de.ewetel.test.DummyServlet doThis_aroundBody4
INFO: Hello
World!
10.03.2006 08:48:48 de.ewetel.test.DummyServlet
doThis_aroundBody4
If you want to replicate my results: its just a
simple servlet that
receives a user request. The doGet()-implementation of
the servlet just
forwards the request to a method (doThis()) that opens a
connection to a
remote EJB object (located in Bea Weblogic) and calls a
HelloWorld-method ;) On each request you get the output i pasted
before.
if you have any more ideas how to reduce the overhead or need
someone to
test a new overhead-reducing aspectj version please let me know
;)
Greetins,
Thilo
Ron Bodkin wrote:
>Hi
Thilo,
>
>I think the results are encouraging in the sense that my
optimizations have
>a big positive impact (1/6 the overhead with RMI,
and at most 1/3 without
>RMI). As Andy and I noted, we're working to
integrate them into AspectJ
>HEAD, in addition to the optimizations that
Andy and Adrian have been
>working on.
>
>While your tests
showed while the overhead is significantly reduced, there
>is still some
noticeable overhead per ClassLoader. I'd like to optimize it
>further to
allow nearly all the state in a case like this to be
shared.
>
>Java VM's create ClassLoaders for RMI calls and
proxies, presumably to
>isolate them so that "normal" classes don't see
them. In this and in other
>circumstances, Java libraries often assume
creating a ClassLoader is a
>lightweight operation... So I think AspectJ
load-time weaving needs to keep
>focusing on adding small overhead in
these cases.
>
>I do think there's an interesting short-term
answer to scenarios like this.
>It would be helpful if the aop.xml file
allowed excluding weaving in child
>ClassLoaders by name pattern,
e.g.,
><aspectj>
> <weaver>
>
<exclude loader="sun.reflect.DelegatingClassLoader"/>
>
</weaver>
>...
></aspectj>
>
>While
this isn't ideal, it would make a huge difference in the
runtime
>performance of load-time weaving (as in cases like this where
you just
don't
>need to weave into the generated RMI
classes).
>
>I'd be interested in trying to replicate your results
in a simple example.
>Am I right that you are just opening a connection
to a remote EJB object
>from inside your Web
application?
>
>Integrating the optimizations for shared state is
still pending some
>foundational work that's in progress...
>
>One thing that should help a lot is making sure that there are
no hard
>references to the weaving state: conceptually whatever memory
the weaver is
>allocating for this ClassLoader should be eligible for GC
once the RMI
proxy
>object is eligible for GC... so some of the
recent work in AspectJ CVS HEAD
>might just fix your memory problem. Of
course, creating a megabyte of
>weaving state each time you use an RMI
proxy is probably going to have some
>negative performance
impact...
>
>-----Original Message-----
>From:
aspectj-users-bounces@xxxxxxxxxxx
>[mailto:aspectj-users-bounces@xxxxxxxxxxx]
On Behalf Of Thilo Focke
>Sent: Monday, March 06, 2006 1:44
AM
>To: aspectj-users@xxxxxxxxxxx
>Subject: Re:
[aspectj-users] problems with load time weaving and
rmi
>communication
>
>Hi guys,
>
>first of all
thank you for your answers.
>i did some testing with the aspectjweavers
from aspectj final, from the
>current development build and from Rods
expermintal build from glassbox
>inspector. The results are alarmingly
different.
>
>i did a little stress tests, that did about 3000
requests in a loop.
without
>rmi rmi communication there is no
significant memory usage. with rmi
>communication and rods development
aspectjweaver i was able to send nearly
>3000 requests. the memory usage
went up to 500 MByte. with the
aspectjweaver
>from the aspectj final
build and from the actual development build i was
>only able to send
nearly 500 requests. the memory usage went up to 500MByte
>and i got an
out of memory error. rods experimental build made
a
significant
>difference in memory usage.
>
>i
summarized my results on the following
url:
>http://www.webengineer.de/ltw.html
>there you can find some
screenhots from jconsole i made during my tests.
>
>i also
switched on the -verbose option in my aop.xml. as matthew
mentioned
>before, when using rmi communication there are created a lot
of class
>loaders. see the following i pasted from my tomcat
console:
>
>06.03.2006 10:17:36 myTest.DummyServlet
doThis_aroundBody4
>info AspectJ Weaver Version LTWExperimental built on
Thursday Jan 26, 2006
>at 03:39:19 GMT
>info register
classloader
>weblogic.utils.classloaders.GenericClassLoader@18880287
>info
using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info
register aspect de.pmon.PerformanceMonitor
>info weaving
'myTest/bean/Service_ee9ogc_HomeImpl_814_WLStub'
>info AspectJ Weaver
Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19
GMT
>info register
classloader
>weblogic.utils.classloaders.GenericClassLoader@32211177
>info
using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info
register aspect de.pmon.PerformanceMonitor
>info weaving
'myTest/bean/Service_ee9ogc_EOImpl_814_WLStub'
>06.03.2006 10:17:36
myTest.DummyServlet doThis_aroundBody4
>06.03.2006 10:17:36
myTest.DummyServlet doThis_aroundBody4
>info AspectJ Weaver Version
LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19
GMT
>info register
classloader
>weblogic.utils.classloaders.GenericClassLoader@20872473
>info
using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info
register aspect de.pmon.PerformanceMonitor
>info weaving
'myTest/bean/Service_ee9ogc_HomeImpl_814_WLStub'
>info AspectJ Weaver
Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19
GMT
>info register
classloader
>weblogic.utils.classloaders.GenericClassLoader@22373430
>info
using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info
register aspect de.pmon.PerformanceMonitor
>info weaving
'myTest/bean/Service_ee9ogc_EOImpl_814_WLStub'
>06.03.2006 10:17:36
myTest.DummyServlet doThis_aroundBody4
>06.03.2006 10:17:36
myTest.DummyServlet doThis_aroundBody4
>info AspectJ Weaver Version
LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19
GMT
>info register
classloader
>weblogic.utils.classloaders.GenericClassLoader@25411170
>info
using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info
register aspect de.pmon.PerformanceMonitor
>info weaving
'myTest/bean/Service_ee9ogc_HomeImpl_814_WLStub'
>info AspectJ Weaver
Version LTWExperimental built on Thursday Jan 26, 2006
>at 03:39:19
GMT
>info register
classloader
>weblogic.utils.classloaders.GenericClassLoader@10284000
>info
using
>file:/C:/Programme/apache-tomcat-5.5.12/shared/lib/pmon-1.0.0-SNAPSHOT.jar!
/
>META-INF/aop.xml
>info
register aspect de.pmon.PerformanceMonitor
>info weaving
'myTest/bean/Service_ee9ogc_EOImpl_814_WLStub'
>06.03.2006 10:17:36
myTest.DummyServlet doThis_aroundBody4
>06.03.2006 10:17:36
myTest.DummyServlet doThis_aroundBody4
>
>can you explain me why
it is necessary to register a new classloader on
each
>rmi
request?
>we planned to integrate the permormance monitoring in one of
our production
>systems. but as long as the memory increases on each rmi
request we think
it
>is not a good idea to integrate it. currently we
only use it on our test
>systems
(unfortunatley).
>
>greetings,
>thilo
>
>
>
>
>Ron
Bodkin wrote:
>
>
>
>>Thilo,
>>
>>Just as a quick follow
up, the AspectJ 1.5.0 release adds some nontrivial
>>overhead per
ClassLoader where you do weaving. When you are using RMI, the
>>VM
can create a lot of different ClassLoaders for reflective
acessors.
>>
>>The experimental LTW version I pointed you at
allows for sharing aspect
>>definitions and AspectJ delegates between
ClassLoaders where it is safe
(it
>>tests for nondelegation), so
it should make a significant difference. I
>>haven't yet created a
patch for this optimization, since as Andy noted,
I'm
>>working
with him to integrate some foundational memory
improvements,
>>bringing them up to generally usable production
quality :-)
>>
>>The same issue applies for JSP's, where
many containers create a large
>>number of lightweight ClassLoaders
and maintaining heavyweight weaving
>>
>>
>state
>
>
>>for each
results in major overhead.
>>
>>-----Original
Message-----
>>From:
aspectj-users-bounces@xxxxxxxxxxx
>>[mailto:aspectj-users-bounces@xxxxxxxxxxx]
On Behalf Of Thilo Focke
>>Sent: Friday, March 03, 2006 8:06
AM
>>To: aspectj-users@xxxxxxxxxxx
>>Subject: Re:
[aspectj-users] problems with load time weaving and
rmi
>>communication
>>
>>Hi Matthew and
Andy,
>>
>>1. The memory usage when using RMI without the
aspect ist absolutely
>>constant and in acceptable ranges (about
30-40 MByte). The memory usage
>>increases permanently when using the
aspect. When using the aspect on code
>>without RMI communication the
memory usage is constant (about 100MByte).
>>
>>2. IŽm using
AspectJ 1.5.0, released on 20th December, 2005
>>
>>3.
My aop.xml looks like this
>>
>><!DOCTYPE aspectj PUBLIC
"-//AspectJ//DTD//EN"
>>"http://www.aspectj.org/dtd/aspectj_1_5_0.dtd">
>><aspectj>
>>
<weaver options="-proceedOnError
-noWarn
>>-Xlintfile:META-INF/Xlint.properties">
>>
<include within="my.package..*"/>
>> </weaver>
>> <aspects>
>> <aspect
name="MyMonitor"/>
>>
</aspects>
>></aspectj>
>>
>>Thanks a
lot,
>>Thilo
>>
>>
>>
>>
>>--------------->
>>
>>Thilo,
>>
>>Can
I ask some questions:
>>1. What is the memory usage when you use RMI
but not the aspect?
>>2. Which version of AspectJ are you using? We
have made some enhancements
>>recently in memory
usage.
>>3. Do you use -verbose in the weaver options of aop.xml? If
so could you
>>post it. I think the problem might to do with
creating lots of class
>>loaders, each with a weaver and the
accompanying
footprint.
>>
>>Cheers
>>
>>Matthew
Webster
>>AOSD Project
>>Java Technology Centre,
MP146
>>IBM Hursley Park, Winchester, SO21
2JN, England
>>Telephone: +44
196 2816139 (external) 246139 (internal)
>>Email: Matthew
Webster/UK/IBM @ IBMGB,
matthew_webster@xxxxxxxxxx
>>_______________________________________________
>>aspectj-users
mailing list
>>aspectj-users@xxxxxxxxxxx
>>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>>
>>
>>_______________________________________________
>>aspectj-users
mailing list
>>aspectj-users@xxxxxxxxxxx
>>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>>
>>
>>
>
>_______________________________________________
>aspectj-users
mailing list
>aspectj-users@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>
>_______________________________________________
>aspectj-users
mailing list
>aspectj-users@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/aspectj-users
>
>
>
--
EWE TEL GmbH
Thilo Focke
IP/IT-Systeme
& Netze
Cloppenburger Strasse 310, D-26133 Oldenburg
Telefon: +49
(0)441 8000-2863
Telefax: +49 (0)441
8000-2799
thilo.focke@xxxxxxxxx
http://www.ewetel.de
_______________________________________________
aspectj-users
mailing list
aspectj-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/aspectj-users
_______________________________________________
aspectj-users
mailing list
aspectj-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/aspectj-users_______________________________________________
aspectj-users mailing list
aspectj-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/aspectj-users
_______________________________________________