Community
Participate
Working Groups
ArrayIndexOutOfBoundsException occurs when compiling a very large codebase with Aspects. We have ~17k files and ~200+ aspects. Memory usage is a bit egregious also hitting around 1GB before throwing this exception. java.lang.ArrayIndexOutOfBoundsException: -1 at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkInheritedMethods(MethodVerifier15.java:348) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkMethods(MethodVerifier15.java:468) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:739) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.verify(MethodVerifier15.java:738) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.verifyMethods(SourceTypeBinding.java:1703) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.verifyMethods(CompilationUnitScope.java:776) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:622) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:392) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performCompilation(AjBuildManager.java:995) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.doBuild(AjBuildManager.java:269) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.batchBuild(AjBuildManager.java:184) at org.aspectj.ajde.core.internal.AjdeCoreBuildManager.doBuild(AjdeCoreBuildManager.java:95) at org.aspectj.ajde.core.AjCompiler.build(AjCompiler.java:118) at org.eclipse.ajdt.core.builder.AJBuilder.build(AJBuilder.java:198) at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:633) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:170) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:201) at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:253) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:256) at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:309) at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:341) at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:140) at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:238) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
I will take a look at this today. Can you tell me what version of AspectJ you are on, is it the release candidate for 1.6.1rc1? (If you are running on a dev build of AJDT on Eclipse 3.3, then yes it is) I presume you have not turned off pipelining compilation?
hmm, this is going to be extremely hard to diagnose without a test case.
(In reply to comment #1) > I will take a look at this today. Can you tell me what version of AspectJ you > are on, is it the release candidate for 1.6.1rc1? (If you are running on a dev > build of AJDT on Eclipse 3.3, then yes it is) Eclipse Version: 3.4.0 Build id: I20080617-2000 AspectJ Runtime/Weaver 1.6.1.20080624142100 > > I presume you have not turned off pipelining compilation? Correct. I presume that it is on by default and that I would need to pass -Xset:pipelineCompilation=false to actually turn if off? >
(In reply to comment #2) > hmm, this is going to be extremely hard to diagnose without a test case. > I have a consistently reproducible test case but it isn't anything I can send you. If you have debug classes that you would like me to install and test with I would be happy to do so. Let me know :)
Although compilation modifications have made the memory footprint "significantly" better I'm wondering if there is anything that can be done with the total memory footprint? Running under Eclipse 3.3 and compiling aspects into our product produced a total memory footprint of ~2GB at which time my desktop became to slow to use. Running Eclipse 3.4 and the latest AJDT I am getting the ArrayIndexOutOfBounds sited in this bug, at around 1GB of memory usage. Running jmap against the 1 GB usage show the following output: Object Histogram: Size Count Class description ------------------------------------------------------- 132978736 2314813 char[] 73239864 1892066 java.lang.Object[] 63780528 569469 org.aspectj.weaver.ReferenceType 46273592 340247 org.aspectj.weaver.TypeVariableReferenceType 44640624 1860026 java.util.ArrayList 39505088 121822 org.aspectj.org.eclipse.jdt.internal.compiler.ast.JavadocSingleNameReference[] 39500784 1645866 java.lang.String 34516928 108358 int[] 18140216 401232 long[] 17120440 428011 org.aspectj.org.eclipse.jdt.internal.compiler.ast.SingleTypeReference 14866592 98129 * ConstMethodKlass 14044960 852128 org.aspectj.weaver.ResolvedType[] 13611360 340284 org.aspectj.weaver.TypeVariable 13379784 152043 org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodScope 11667344 112186 org.aspectj.ajdt.internal.compiler.ast.AjMethodDeclaration 10887936 340248 org.aspectj.weaver.BoundedReferenceType$ReferenceTypeReferenceTypeDelegate 10306624 161041 org.aspectj.org.eclipse.jdt.internal.compiler.ast.Argument We are piloting the use of aspects in our product but the total memory footprint may prevent us from doing so. Thoughts? Thanks Joe
> I have a consistently reproducible test case but it isn't anything I can send > you. > > If you have debug classes that you would like me to install and test with I > would be happy to do so. Thanks for the offer - I can send you a debug compiler but it is unlikely to be something I can solve with the first debug build, we might need a few iterations... 1.6.1 was aimed mostly at weaving memory usage rather than compiler memory usage (to aid loadtime weaving, binary weaving and the back end of the compilation process). However, there is a change discussed in bug 227484 which might benefit your situation, but I haven't put it in the dev builds yet as it needs a lot more testing - but i could possibly include it with debug drivers I send you. Your jmap output shows roughly what I would expect to see, although I didn't spend any time optimizing memory usage of compile time structures during 1.6.1.
(In reply to comment #6) > > I have a consistently reproducible test case but it isn't anything I can send > > you. > > > > If you have debug classes that you would like me to install and test with I > > would be happy to do so. > > Thanks for the offer - I can send you a debug compiler but it is unlikely to be > something I can solve with the first debug build, we might need a few > iterations... Not a problem, I do this for a living also. My crystal ball broke a few years back so I have to do it the old fashion way, iteratively ;) > > 1.6.1 was aimed mostly at weaving memory usage rather than compiler memory > usage (to aid loadtime weaving, binary weaving and the back end of the > compilation process). However, there is a change discussed in bug 227484 which > might benefit your situation, but I haven't put it in the dev builds yet as it > needs a lot more testing - but i could possibly include it with debug drivers I > send you. Works for me. If it's not too large you can e-mail it to me at liquid@visi.com or my work e-mail jvandemark@ptc.com. jar/tar/zip are all fine. > > Your jmap output shows roughly what I would expect to see, although I didn't > spend any time optimizing memory usage of compile time structures during 1.6.1. > Thanks for the time Joe
Ok, I have a first new build to try out. You mentioned you are in eclipse, so this is a patch for AJDT on Eclipse 3.4. I've done 3 things at once (possibly not smart, but we'll see). I think this bug is due to the use of intertype declarations and the code in the methodverifier that comes with the standard JDT does get confused by those sometimes. I have: 1. Made that chunk of code a little more aware of intertype declarations, this *might* alleviate the problem 2. Added debug just at the point where we would normally use an invalid array accessor. Debug is to standard error, so might appear in the window in which you launched eclipse from. (If you normally launch from a shortcut, try launching from a command prompt to see it) 3. When the situation happens I print the debug then attempt to avoid failing. This may show us if there is another problem down the line. The jar file is ajde_pr239120_1.jar available here: http://www.eclipse.org/downloads/download.php?file=/tools/aspectj/dev/ajde_pr239120_1.jar It is 5.3M so I didn't email it - unfortunately there is a delay from when I upload things to eclipse as they replicate around the servers for download. I tried just now and that link isnt live yet, but it should be soon. That jar is a replacement for ajde.jar within the eclipse org.aspectj.ajde plugin (do you know where the plugins are for patching?) Applying the patch (by replacing the existing jar) may be complicated by the existence of the dreaded 'P2' in eclipse 3.4, which might not pick up the new jar - I don't have enough P2 knowledge yet to know how to get round it. You should try a straight patch of that plugin and try a restart of eclipse. If that doesn't work we can try and kick P2 into shape - or what would be better is if you could do a build from the command line and see the same problem. Then I can more easily give you patches that don't involve eclipse, I'll just give you new versions of the compiler to run. let me know how you get on. When we get past this, we can look at memory usage.
(In reply to comment #8) > Ok, I have a first new build to try out. You mentioned you are in eclipse, so > this is a patch for AJDT on Eclipse 3.4. I've done 3 things at once (possibly > not smart, but we'll see). I think this bug is due to the use of intertype > declarations and the code in the methodverifier that comes with the standard > JDT does get confused by those sometimes. > > I have: > > 1. Made that chunk of code a little more aware of intertype declarations, this > *might* alleviate the problem > 2. Added debug just at the point where we would normally use an invalid array > accessor. Debug is to standard error, so might appear in the window in which > you launched eclipse from. (If you normally launch from a shortcut, try > launching from a command prompt to see it) > 3. When the situation happens I print the debug then attempt to avoid failing. > This may show us if there is another problem down the line. > > The jar file is ajde_pr239120_1.jar available here: > > http://www.eclipse.org/downloads/download.php?file=/tools/aspectj/dev/ajde_pr239120_1.jar > > [ snip ] > > let me know how you get on. When we get past this, we can look at memory > usage. > I installed the new jar by means of a symbolic link (rm-d the original jar) verified the installations: josepy@stealth /opt/gme/eclipse $ cd plugins /opt/gme/eclipse/plugins josepy@stealth /opt/gme/eclipse/plugins $ find . -name 'ajde.jar' ./org.aspectj.ajde_1.6.1.20080624142100/ajde.jar josepy@stealth /opt/gme/eclipse/plugins $ ls -al ./org.aspectj.ajde_1.6.1.20080624142100/ajde.jar lrwxrwxrwx 1 josepy josepy 19 2008-07-03 08:06 ./org.aspectj.ajde_1.6.1.20080624142100/ajde.jar -> ajde_pr239120_1.jar I ran Eclipse from a shell as follows: ./eclipse 2>&1 | tee ajde_pr239120_1.log Tested twice the second time using a -verbose to verify class loading with the following result: [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.flow.LabelFlowContext from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.codegen.DoubleCache from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.ast.LabeledStatement from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.codegen.FloatCache from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.util.SimpleSet from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.ast.DoStatement from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] java.lang.NullPointerException at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkConcreteInheritedMethod(MethodVerifier15.java:109) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.checkInheritedMethods(MethodVerifier.java:289) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkInheritedMethods(MethodVerifier15.java:374) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkMethods(MethodVerifier15.java:489) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:739) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.verify(MethodVerifier15.java:759) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.verifyMethods(SourceTypeBinding.java:1703) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.verifyMethods(CompilationUnitScope.java:776) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:622) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:392) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performCompilation(AjBuildManager.java:995) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.doBuild(AjBuildManager.java:269) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.batchBuild(AjBuildManager.java:184) at org.aspectj.ajde.core.internal.AjdeCoreBuildManager.doBuild(AjdeCoreBuildManager.java:95) at org.aspectj.ajde.core.AjCompiler.build(AjCompiler.java:118) at org.eclipse.ajdt.core.builder.AJBuilder.build(AJBuilder.java:198) at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:633) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:170) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:201) at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:253) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:256) at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:309) at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:341) at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:140) at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:238) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) [Loaded org.eclipse.ajdt.internal.ui.ajde.AJDTErrorHandler from file:/opt/gme/eclipse/plugins/org.eclipse.ajdt.ui_1.6.0.200806271302.jar] [Loaded org.eclipse.ajdt.internal.ui.ajde.AJDTErrorHandler$1 from file:/opt/gme/eclipse/plugins/org.eclipse.ajdt.ui_1.6.0.200806271302.jar] I didn't note any debug in either run. The stack looks to be a couple frames deeper and is produced via a NullPointerException. Hope this gives you a little more to go on. Let me know how you want to proceed.
Created attachment 106517 [details] Class file patch for previously mentioned ajde jar This zip file contains a new version of MethodVerifier15. It should be applied to the ajde.jar previously attached to this patch. To apply it, just do something like the following (in the directory where the patched ajde.jar currently resides) jar -xvf patch_pr239120_2.zip jar -uvf ajde_pr239120_1.jar org There was a bug in my debug logic (*sigh*) which may have led to the NPE, this change addresses that bug.
I'm glad it was relatively straightforward to get eclipse to see the new ajde jar. I was going to upload a new ajde.jar but just patching one class file seemed faster this time.
(In reply to comment #11) > I'm glad it was relatively straightforward to get eclipse to see the new ajde > jar. I was going to upload a new ajde.jar but just patching one class file > seemed faster this time. > Updated jar Installed & verified install: josepy@stealth /opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100 $ pwd /opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100 josepy@stealth /opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100 $ ls -al total 5400 drwxr-xr-x 5 josepy josepy 4096 2008-07-04 10:33 ./ drwxr-xr-x 25 josepy josepy 24576 2008-07-03 13:44 ../ -rw-r--r-- 1 josepy josepy 1385 2008-06-25 19:58 about.html lrwxrwxrwx 1 josepy josepy 19 2008-07-04 10:33 ajde.jar -> ajde_pr239120_1.jar -rw-r--r-- 1 josepy josepy 2041 2008-06-25 19:58 ajde.jardesc -rw-r--r-- 1 josepy josepy 5417988 2008-07-04 10:30 ajde_pr239120_1.jar drwxr-xr-x 13 josepy josepy 4096 2008-06-27 06:50 doc/ drwxr-xr-x 4 josepy josepy 4096 2008-06-27 06:50 icons/ drwxr-xr-x 2 josepy josepy 4096 2008-06-27 06:50 META-INF/ -rw-r--r-- 1 josepy josepy 574 2008-06-25 19:58 plugin.properties -rw-r--r-- 1 josepy josepy 464 2008-06-25 19:58 plugin.xml -rw-r--r-- 1 josepy josepy 8774 2008-06-25 19:58 toc_adk15notebook.xml -rw-r--r-- 1 josepy josepy 6505 2008-06-25 19:58 toc_ajdevguide.xml -rw-r--r-- 1 josepy josepy 2113 2008-06-25 19:58 toc_ajpdguide.xml -rw-r--r-- 1 josepy josepy 9115 2008-06-25 19:58 toc_ajprogguide.xml -rw-r--r-- 1 josepy josepy 987 2008-06-25 19:58 toc.xml josepy@stealth /opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100 $ jar -tvf ajde_pr239120_1.jar | grep "MethodVerifier 40897 Wed Jul 02 12:13:42 CDT 2008 org/aspectj/org/eclipse/jdt/internal/compiler/lookup/MethodVerifier.class 42010 Thu Jul 03 13:21:38 CDT 2008 org/aspectj/org/eclipse/jdt/internal/compiler/lookup/MethodVerifier15.class josepy@stealth /opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100 $ jar -tvf ajde.jar | grep "MethodVerifier*" 40897 Wed Jul 02 12:13:42 CDT 2008 org/aspectj/org/eclipse/jdt/internal/compiler/lookup/MethodVerifier.class 42010 Thu Jul 03 13:21:38 CDT 2008 org/aspectj/org/eclipse/jdt/internal/compiler/lookup/MethodVerifier15.class Unfortunately we get the exact same stack trace .. [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] [Loaded org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15 from file:/opt/gme/eclipse/plugins/org.aspectj.ajde_1.6.1.20080624142100/ajde.jar] java.lang.NullPointerException at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkConcreteInheritedMethod(MethodVerifier15.java:109) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.checkInheritedMethods(MethodVerifier.java:289) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkInheritedMethods(MethodVerifier15.java:374) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkMethods(MethodVerifier15.java:489) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:739) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.verify(MethodVerifier15.java:759) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.verifyMethods(SourceTypeBinding.java:1703) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.verifyMethods(CompilationUnitScope.java:776) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:622) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:392) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.performCompilation(AjBuildManager.java:995) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.doBuild(AjBuildManager.java:269) at org.aspectj.ajdt.internal.core.builder.AjBuildManager.batchBuild(AjBuildManager.java:184) at org.aspectj.ajde.core.internal.AjdeCoreBuildManager.doBuild(AjdeCoreBuildManager.java:95) at org.aspectj.ajde.core.AjCompiler.build(AjCompiler.java:118) at org.eclipse.ajdt.core.builder.AJBuilder.build(AJBuilder.java:198) at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:633) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:170) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:201) at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:253) at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37) at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:256) at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:309) at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:341) at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:140) at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:238) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) Let me know when you have something new to try, or if there is anything else I can do to help :) BTW happy 4th
(In reply to comment #12) If you send along the source files in the jar I can try to debug it ... Just a thought.
Created attachment 106609 [details] source for most recent patch class Hi Joe, Unfortunately I'm travelling for the next day or so and won't be able to put together another patch. For the first patch I changed a few files and so uploaded the zip. I've got time to attach the source for the most recent variant of MethodVerifier15 but not for the full ajde (i gotta pack!). The ajde src zip already available in AJDT in the org.aspectj.ajde plugin will be mostly correct (99%) - I've really only modified MethodVerifier15 (and then some minor changes to the ProblemReporter hierarchy). So you should be able to use that source jar with this new .java file and be able to sensibly debug. MethodVerifier15 changes I have made so far: - where the count is reduced within the original failing method, I changed the error reporting call such that it returned a boolean indicated whether the error actually got reported. (it will not get reported for ITDs). If the error is not reported i do not reduce the count or do the associated nulling. This ought to stop the count and length counting out of sync (and as such the incorrect -1 index into the array at the end) - changed it such that if the -1 situation arises, we don't incorrectly index the array and dump out the debug. It looks like the first case is correctly preventing the -1 failure. It would be interesting to know if: - the changes I made are causing us to fail earlier than we used to (verifying some other earlier method) - what is the full set of methods in the hierarchy subject to verification at the time of the failure, are there ITDs in there. To determine if the former is happening, you could use the original org.aspectj.ajde plugin and debug to see what method we fail to verify, then apply the patch and see what method we now fail to verify upon. And by determining the pattern of methods in the mix when the failure occurs, it should be possible to create a small testcase for the problem that mirrors the same ITD setup. There are very likely to be other places in the MethodVerifier (and MethodVerifier15) code that are not aware of ITDs and need to be made aware.
[snip] > MethodVerifier15 changes I have made so far: > > - where the count is reduced within the original failing method, I changed the > error reporting call such that it returned a boolean indicated whether the > error actually got reported. (it will not get reported for ITDs). If the > error is not reported i do not reduce the count or do the associated nulling. > This ought to stop the count and length counting out of sync (and as such the > incorrect -1 index into the array at the end) > - changed it such that if the -1 situation arises, we don't incorrectly index > the array and dump out the debug. > > It looks like the first case is correctly preventing the -1 failure. It would > be interesting to know if: > > - the changes I made are causing us to fail earlier than we used to (verifying > some other earlier method) > - what is the full set of methods in the hierarchy subject to verification at > the time of the failure, are there ITDs in there. > > To determine if the former is happening, you could use the original > org.aspectj.ajde plugin and debug to see what method we fail to verify, then > apply the patch and see what method we now fail to verify upon. > > And by determining the pattern of methods in the mix when the failure occurs, > it should be possible to create a small testcase for the problem that mirrors > the same ITD setup. > > There are very likely to be other places in the MethodVerifier (and > MethodVerifier15) code that are not aware of ITDs and need to be made aware. > Didn't spend a lot of time this weekend working on this but noted a couple of things: When MethodVerifier15.checkInheritedMethods is called the length passed is sometimes less than the number of elements in the MethodBinding array ? don't know if thats a problem or not. In the situation where we are getting a NPE the MethodBindings look like the following: (they include ITDs): >> MethodVerifier15.checkInheritedMethods: type wt.viewmarkup.WVSConfigurationTemplate MethodBinding[2], length passed = 2 >> MethodVerifier15.checkInheritedMethods: type wt.viewmarkup.WVSConfigurationTemplate MethodBinding[3], length passed = 2 >> MethodVerifier15.checkInheritedMethods: type wt.viewmarkup.WVSConfigurationTemplate MethodBinding[3], length passed = 3 method (0) boolean equals(java.lang.Object) method (1) InterTypeMethodBinding(boolean equals(java.lang.Object) , (id=NoId) public interface wt.fc.Persistable extends java.lang.Object implements : wt.fc.ObjectMappable /* fields */ java.lang.String IDENTITY java.lang.String PERSIST_INFO java.lang.String TYPE /* methods */ void checkAttributes() throws wt.fc.InvalidAttributeException java.lang.String getIdentity() java.lang.String getType() /* members */ Member type : IMPL (id=NoId) public static class wt.fc.Persistable$IMPL extends java.lang.Object enclosing type : wt.fc.Persistable /* methods */ void <init>() void ajc$interFieldInit$wt_fc_Persistable$IMPL$wt_fc_Persistable$thePersistInfo(wt.fc.Persistable) boolean ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$equals(wt.fc.Persistable, java.lang.Object) wt.fc.PersistInfo ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$getPersistInfo(wt.fc.Persistable) int ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$hashCode(wt.fc.Persistable) void ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$setPersistInfo(wt.fc.Persistable, wt.fc.PersistInfo) wt.fc.Persistable.IMPL aspectOf() boolean hasAspect() ) method (2) InterTypeMethodBinding(boolean equals(java.lang.Object) , (id=NoId) public interface wt.fc.Persistable extends java.lang.Object implements : wt.fc.ObjectMappable /* fields */ java.lang.String IDENTITY java.lang.String PERSIST_INFO java.lang.String TYPE /* methods */ void checkAttributes() throws wt.fc.InvalidAttributeException java.lang.String getIdentity() java.lang.String getType() /* members */ Member type : IMPL (id=NoId) public static class wt.fc.Persistable$IMPL extends java.lang.Object enclosing type : wt.fc.Persistable /* methods */ void <init>() void ajc$interFieldInit$wt_fc_Persistable$IMPL$wt_fc_Persistable$thePersistInfo(wt.fc.Persistable) boolean ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$equals(wt.fc.Persistable, java.lang.Object) wt.fc.PersistInfo ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$getPersistInfo(wt.fc.Persistable) int ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$hashCode(wt.fc.Persistable) void ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$setPersistInfo(wt.fc.Persistable, wt.fc.PersistInfo) wt.fc.Persistable.IMPL aspectOf() boolean hasAspect() Hope you had a nice 4th Joe
Hi Joe, I am on a trip in the UK at the moment, but am trying to look at this when I can. That debug info ought to be enough to give us a way to recreate. My first go is just to simply try and target a type with multiple ITDs: interface Hoo { } interface Goo extends Hoo { } public class Foo implements Goo { public boolean equals(Object o) { return true; } } aspect X { public boolean Goo.equals(Object o) { return true; } public boolean Hoo.equals(Object o) { return true; } } but it works fine *sigh* - I need to step in with the debugger and see if it recreated a similar kind of scenario in the MethodVerifier15 code.
(In reply to comment #16) > Hi Joe, > > I am on a trip in the UK at the moment, but am trying to look at this when I > can. That debug info ought to be enough to give us a way to recreate. > > My first go is just to simply try and target a type with multiple ITDs: > interface Hoo { > } > > interface Goo extends Hoo { > } > > public class Foo implements Goo { > public boolean equals(Object o) { return true; } > } > > aspect X { > public boolean Goo.equals(Object o) { return true; } > public boolean Hoo.equals(Object o) { return true; } > } > > but it works fine *sigh* - I need to step in with the debugger and see if it > recreated a similar kind of scenario in the MethodVerifier15 code. > Hey Enjoy your vacation, we can deal with this when you get back! I've got plenty of other problems to fill my time ;) Joe
Just FYI. I was able to compile outside of Eclipse and doing so provide some addition information. I see your debug output ... well once anyhow (below). I inserted a wrapper method in place of checkInheritedMethods which calls the original checkInheritedMethods, the new method has a try/catch block as follows: void checkInheritedMethods(MethodBinding[] methods, int length) { try { checkInheritedMethods2 (methods, length); // call original method } catch (Exception e) { e.printStackTrace(); } } The NullPointerException has the follow stack. java.lang.NullPointerException at org.aspectj.ajdt.internal.compiler.lookup.OwningClassSupportForMethodBindings.ajc$interMethodDispatch1$org_aspectj_ajdt_internal_compiler_lookup_OwningClassSupportForMethodBinding at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.declaringClass_aroundBody47$advice(MethodVerifier.java:71) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.findOverriddenInheritedMethods(MethodVerifier.java:620) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.checkInheritedMethods(MethodVerifier.java:240) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkInheritedMethods2(MethodVerifier15.java:392) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkInheritedMethods(MethodVerifier15.java:305) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.checkMethods(MethodVerifier15.java:507) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier.verify(MethodVerifier.java:739) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.MethodVerifier15.verify(MethodVerifier15.java:777) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.verifyMethods(SourceTypeBinding.java:1703) at org.aspectj.org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.verifyMethods(CompilationUnitScope.java:776) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:622) at org.aspectj.org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:392) The long and short seems to be that the MethodBindings[] methods is not getting processed correctly so when it is passed along to super.checkInheritedMethods it has a null member/element somewhere in the middle. I'll keep debugging in my spare time. Ping me when you get back from your trip :) Enjoy Joe > > Hey > > Enjoy your vacation, we can deal with this when you get back! I've got plenty > of other problems to fill my time ;) > > Joe > DEBUG: In pr239120 situation... DEBUG: Count was 2 Length was 4 Methods being checked for this type wt.eff.IncorporationDate Number of methods 4 method (0) boolean equals(java.lang.Object) method (1) null method (2) InterTypeMethodBinding(boolean equals(java.lang.Object) , (id=NoId) public interface wt.fc.Persistable extends java.lang.Object implements : wt.fc.ObjectMappable /* fields */ java.lang.String IDENTITY java.lang.String PERSIST_INFO java.lang.String TYPE /* methods */ void checkAttributes() throws wt.fc.InvalidAttributeException java.lang.String getIdentity() java.lang.String getType() /* members */ Member type : IMPL (id=NoId) public static class wt.fc.Persistable$IMPL extends java.lang.Object enclosing type : wt.fc.Persistable /* methods */ void <init>() void ajc$interFieldInit$wt_fc_Persistable$IMPL$wt_fc_Persistable$thePersistInfo(wt.fc.Persistable) boolean ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$equals(java.lang.Object) wt.fc.PersistInfo ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$getPersistInfo() int ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$hashCode() void ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$setPersistInfo(wt.fc.PersistInfo) wt.fc.Persistable.IMPL aspectOf() boolean hasAspect() ) method (3) InterTypeMethodBinding(boolean equals(java.lang.Object) , (id=NoId) public interface wt.fc.Persistable extends java.lang.Object implements : wt.fc.ObjectMappable /* fields */ java.lang.String IDENTITY java.lang.String PERSIST_INFO java.lang.String TYPE /* methods */ void checkAttributes() throws wt.fc.InvalidAttributeException java.lang.String getIdentity() java.lang.String getType() /* members */ Member type : IMPL (id=NoId) public static class wt.fc.Persistable$IMPL extends java.lang.Object enclosing type : wt.fc.Persistable /* methods */ void <init>() void ajc$interFieldInit$wt_fc_Persistable$IMPL$wt_fc_Persistable$thePersistInfo(wt.fc.Persistable) boolean ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$equals(java.lang.Object) wt.fc.PersistInfo ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$getPersistInfo() int ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$hashCode() void ajc$interMethod$wt_fc_Persistable$IMPL$wt_fc_Persistable$setPersistInfo(wt.fc.PersistInfo) wt.fc.Persistable.IMPL aspectOf() boolean hasAspect() )
Joe - have you had a chance to try this with a recent AspectJ? Since that will include many of the changes I've made for bug 242797 - and in those I directly change the way inherited methods are computed in the MethodVerifier code. I made it so that a method is not inherited (for the purposes of verification) from an interface if there is an ITD on that interface that provides an implementation - in that way the subtypes won't complain (incorrectly) that they don't implement it.
(In reply to comment #19) > Joe - have you had a chance to try this with a recent AspectJ? Since that will > include many of the changes I've made for bug 242797 - and in those I directly > change the way inherited methods are computed in the MethodVerifier code. I > made it so that a method is not inherited (for the purposes of verification) > from an interface if there is an ITD on that interface that provides an > implementation - in that way the subtypes won't complain (incorrectly) that > they don't implement it. > Not yet, hopefully in the next few days.
I noticed this bug in eclipse too. While the project I was running on is very small, there are other aspectj projects open, which might be causing this issue. I upgraded my aspectj plugin and I think I still noticed the bug. I will get back with more details tomorrow. Thanks, -S-
not sure if still an issue, hasn't been reported as coming up since 1.6.1 (which is 7 releases ago). removing target but leaving open as not confirmed fixed.
