Bug 418036 - java.lang.VerifyError JVMVRFY012 stack shape inconsistent
Summary: java.lang.VerifyError JVMVRFY012 stack shape inconsistent
Status: NEW
Alias: None
Product: AspectJ
Classification: Tools
Component: LTWeaving (show other bugs)
Version: 1.7.4   Edit
Hardware: PC AIX
: P3 critical (vote)
Target Milestone: ---   Edit
Assignee: aspectj inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-09-25 11:56 EDT by Alexander Panayotov CLA
Modified: 2014-01-24 04:10 EST (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander Panayotov CLA 2013-09-25 11:56:18 EDT
OS used: AIX 7.1
JRE: 1.7.0 AIX SR5
AspectJ: 1.7.3

java.lang.VerifyError: JVMVRFY012 stack shape inconsistent; class=/***/***/foo, method=bar(Ljava/io/File;)Ljava/lang/String;, pc=128
at java.lang.J9VMInternals.verifyImpl(Native Method)
at java.lang.J9VMInternals.verify(J9VMInternals.java:93)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:170)
...

Over the past 3 days, I have been trying to figure out and eventually fix the so-called "Inconsistent stack shape" issue. Unfortunately, no success so far. 

The weird thing I noticed is that if running with JRE 1.7.0 SR3 and AspectJ 1.6.12, the issue is not reproduced. I tried upgrading to AspectJ 1.7.3, yet no changes are to be noted. 
I believe the issue is located somewhere in the JRE 1.7.0 SR5, which implements some of the Oracle Java Class Libraries, and, according to my recent researches, the so-called "new" byte code verification tool. However, I didn't find anything about this particular Oracle update in the AspectJ update notes. I supposed that upgrading the AspectJ would sort things out, but yet, here I am. I tried working this around with some "solutions" I found like giving extra options to the weaver (-Xnoinline), but nothing helped.

Should you have any questions, do not hesitate to ask.
Comment 1 Andrew Clement CLA 2013-09-25 13:00:31 EDT
Have you tried passing this as a jvm argument (it is a possible workaround): -XX:-UseSplitVerifier
Comment 2 Alexander Panayotov CLA 2013-09-25 16:20:07 EDT
Yes, I have. I just forgot to mention it. I will try to set a test case tomorrow, but I cannot promise anything, since the code is property of a customer.
Comment 3 Alexander Panayotov CLA 2013-09-27 03:43:10 EDT
Apparently, I will not be able to provide a test case. I am aware this makes the things harder. Any suggestions?
Comment 4 Andrew Clement CLA 2013-10-23 11:31:58 EDT
I have seen problems in the past with the J9 VM (when I worked at IBM...) - I recall having to get them to alter the verifier once or twice as it wasn't checking quite the right thing. I can't seem to download a version of it to experiment with (and I certainly haven't access to an AIX system).

Another approach to fixing it is for AspectJ to produce less optimal but more easily verifiable code. But in order to do that I'd need to see the bytecode involved. Can you include the class file that has the bar method?  Based on what I see I might be able to tweak codegen.
Comment 5 Alexander Panayotov CLA 2014-01-10 10:55:26 EST
After dealing with some other stuff, I am finally back to this issue. I would like to apologize for the rather late response, but... well, stuff happens.

Few important notes:
1) it is still reproducible using version 1.7.4 of the weaver
2) Possibly, it is reproducible with IBM JRE 1.7.0 SR6, will confirm next week.
3) I dumped one of the classes in which, according to the stack trace, the bytecode could not be verified, but in the end, the specific method, which was "blacklisted" had exactly the same code before and after the weaving.
4) The issue still occurs on rather random basis, but I was able to get a stable reproduction and because of that perhaps I would be able to provide the dumped bytecode.

Regarding the request for help, in order to see what actually is going on, would you need the bytecode only or the entire dumped classes? In case you need the class files, this might be a bit harder to do because of legal reasons.
Comment 6 Andrew Clement CLA 2014-01-10 12:06:32 EST
I think the bytecode for just the problematic method would be a start, I don't need the whole class - yet (the verbose javap output for the method is ideal). You can email it to aclement AT gopivotal.com rather than posting it here if you'd prefer.
Comment 7 Alexander Panayotov CLA 2014-01-13 03:41:24 EST
I will send you the dumped bytecode for the entire class, so you could get a better understanding and, of course, more information. Just a note - it will be sent via internal secure messenger due to confidentiality reasons.
Comment 8 Alexander Panayotov CLA 2014-01-13 09:25:37 EST
Secondary note: the bytecode dump will be delayed (hopefully with a day at most) since it has to undergo confidentiality check. Will be sent as soon as possible.
Comment 9 Alexander Panayotov CLA 2014-01-24 04:10:41 EST
A brief summary after further investigation:

The issue is NOT reproducible using IBM JRE 1.7.0 SR6. This applies for AspectJ versions 1.6.12 and 1.7.4 (I believe it applies for all versions in between as well). 


In conclusion, it seems that IBM have messed up their JVM in IBM JRE 1.7.0 SR5 and now when SR6 is released, people should update immediately due to important security fixes introduced in the latest release.

Consider this case closed.