Bug 159631 - [dom] NPE while trying to cleanup specific pair of file
Summary: [dom] NPE while trying to cleanup specific pair of file
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.2.1   Edit
Hardware: PC Windows XP
: P3 major (vote)
Target Milestone: 3.2.2   Edit
Assignee: Maxime Daniel CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-10-03 14:56 EDT by Ricky Ng-Adam CLA
Modified: 2007-01-16 05:25 EST (History)
1 user (show)

See Also:


Attachments
screenshots of cleanup configuration and results (203.22 KB, application/x-zip-compressed)
2006-10-03 14:58 EDT, Ricky Ng-Adam CLA
no flags Details
the eclipse log (9.43 KB, application/octet-stream)
2006-10-03 14:58 EDT, Ricky Ng-Adam CLA
no flags Details
another eclipse log extract (slightly different stack) (9.03 KB, application/octet-stream)
2006-10-05 17:42 EDT, Ricky Ng-Adam CLA
no flags Details
A sample debug options file (2.34 KB, text/plain)
2006-10-23 11:02 EDT, Maxime Daniel CLA
no flags Details
Output I get when running Eclipse / Clean Up... as instructed (108.42 KB, text/plain)
2006-10-23 12:09 EDT, Ricky Ng-Adam CLA
no flags Details
Suggested fix - HEAD based (2.16 KB, patch)
2006-10-25 10:05 EDT, Maxime Daniel CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ricky Ng-Adam CLA 2006-10-03 14:56:32 EDT
While trying to refactor our large code base, the Cleanup wizard fails with "An unexpected exception occurred while creating a change object. See the error log for more details" in the Found problems page.

After investigation, It fails when 2 very specific packages (our core.enumeration, core.event.bo) are selected for cleanup; however, I can't seem to isolate it further.  I've tried just copying those two packages in another project but can't reproduce the behavior. I can cleanup each of these package individually.

Will the log file and the list of settings be sufficient to isolate this? I'm attaching those to this bug report.
Comment 1 Ricky Ng-Adam CLA 2006-10-03 14:58:05 EDT
Created attachment 51367 [details]
screenshots of cleanup configuration and results
Comment 2 Ricky Ng-Adam CLA 2006-10-03 14:58:57 EDT
Created attachment 51369 [details]
the eclipse log
Comment 3 Martin Aeschlimann CLA 2006-10-05 11:45:06 EDT
That looks like a problem when building an the AST.

The more you can find out where this happens the better. Did you try to invoke 'clean up' on every compilation unit to find out which one has the problem?

Maybe this is releated to bug 156352.

Caused by: java.lang.NullPointerException
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.storedAnnotations(SourceTypeBinding.java:1434)
at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.storeAnnotationHolder(ReferenceBinding.java:1173)
at org.eclipse.jdt.internal.compiler.lookup.MethodBinding.<init>(MethodBinding.java:61)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.getUpdatedMethodBinding(SourceTypeBinding.java:1074)
at org.eclipse.jdt.internal.compiler.ast.MessageSend.manageSyntheticAccessIfNecessary(MessageSend.java:226)
at org.eclipse.jdt.internal.compiler.ast.MessageSend.analyseCode(MessageSend.java:63)
at org.eclipse.jdt.internal.compiler.ast.IfStatement.analyseCode(IfStatement.java:65)
at org.eclipse.jdt.internal.compiler.ast.Block.analyseCode(Block.java:40)
at org.eclipse.jdt.internal.compiler.ast.WhileStatement.analyseCode(WhileStatement.java:115)
at org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.analyseCode(MethodDeclaration.java:84)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.internalAnalyseCode(TypeDeclaration.java:673)
at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.analyseCode(TypeDeclaration.java:256)
at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.analyseCode(CompilationUnitDeclaration.java:106)
at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:597)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:687)
at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:464)
at org.eclipse.jdt.core.dom.ASTParser.createASTs(ASTParser.java:707)
	at org.eclipse.jdt.internal.corext.fix.CleanUpRefactoring.parse(CleanUpRefactoring.java:518)
Comment 4 Ricky Ng-Adam CLA 2006-10-05 17:40:58 EDT
I just tried isolating the problem further and I'm down to two classes (our EnmEventTimeType and ObjEventTime).  I've tried copying just these two classes to another project and doing "Create class" / "Create method" on dependencies until it compiled but still can't reproduce the exception. I then tried making these created methods more similar to the real thing but it didn't help.  The problem seems rather subtle...

However, the exception I'm getting now seems to follow a different path in the Cleanup code:

As reported in "cleanup failure.log":

Caused by: java.lang.NullPointerException
	at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.storedAnnotations(SourceTypeBinding.java:1434)
	at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.storeAnnotationHolder(ReferenceBinding.java:1173)
	at org.eclipse.jdt.internal.compiler.lookup.MethodBinding.<init>(MethodBinding.java:61)
	at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.getUpdatedMethodBinding(SourceTypeBinding.java:1074)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.manageSyntheticAccessIfNecessary(MessageSend.java:226)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.analyseCode(MessageSend.java:63)
	at org.eclipse.jdt.internal.compiler.ast.IfStatement.analyseCode(IfStatement.java:65)
	at org.eclipse.jdt.internal.compiler.ast.Block.analyseCode(Block.java:40)
	at org.eclipse.jdt.internal.compiler.ast.WhileStatement.analyseCode(WhileStatement.java:115)

Now:

Caused by: java.lang.NullPointerException
	at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.storedAnnotations(SourceTypeBinding.java:1434)
	at org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding.storeAnnotationHolder(ReferenceBinding.java:1173)
	at org.eclipse.jdt.internal.compiler.lookup.MethodBinding.<init>(MethodBinding.java:61)
	at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.getUpdatedMethodBinding(SourceTypeBinding.java:1074)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.manageSyntheticAccessIfNecessary(MessageSend.java:226)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.analyseCode(MessageSend.java:63)
	at org.eclipse.jdt.internal.compiler.ast.AND_AND_Expression.analyseCode(AND_AND_Expression.java:59)
	at org.eclipse.jdt.internal.compiler.ast.IfStatement.analyseCode(IfStatement.java:65)
	at org.eclipse.jdt.internal.compiler.ast.MethodDeclaration.analyseCode(MethodDeclaration.java:84)
Comment 5 Ricky Ng-Adam CLA 2006-10-05 17:42:00 EDT
Created attachment 51505 [details]
another eclipse log extract (slightly different stack)
Comment 6 Ricky Ng-Adam CLA 2006-10-12 17:17:14 EDT
Note also that with absolutely no refactoring ie 0 of 29 selected (funny that I can do that without an error message...) I get the same exception as comment #4.
Comment 7 Maxime Daniel CLA 2006-10-20 03:11:27 EDT
We'd like to deploy an instrumented version of jdt core on your machine to narrow down the issue, if this is possible on your side. I'll prepare one in any case. I assume that you're using Eclipse 3.2.1, that is v_677_R32x of org.eclipse.jdt.core. Please let me know if it is not the case.
Comment 8 Ricky Ng-Adam CLA 2006-10-20 09:23:06 EDT
Yes, that is the version I'm using; 3.2.1.r321_v20060905.  I'd be happy to try your instrumented version.
Comment 9 Maxime Daniel CLA 2006-10-23 11:01:23 EDT
(In reply to comment #8)
> Yes, that is the version I'm using; 3.2.1.r321_v20060905.  I'd be happy to try
> your instrumented version.
> 
Thanks a lot. I have posted an instrumented version of JDT Core at http://www.eclipse.org/jdt/core/patches/org.eclipse.jdt.core_3.2.1.z20061023-1512.jar
Dropping that file into your eclipse/plugins directory should enable some more printing when you reproduce the problematic scenario. Please collect both standard output and standard error streams. You may consider adding some debug tracing by using the -debug <debug options file> on the eclipse launch command - beware though that traces can grow pretty large. I will attach a typical debug options file to this bug, in which I have turned on a few significant ones.
(Please note that this is an unofficial, unsupported version of JDT Core and that you should revert to the supported version for normal operation.)

We are interested in any information that this instrumented version can collect. 

Thanks again for your help.
Comment 10 Maxime Daniel CLA 2006-10-23 11:02:50 EDT
Created attachment 52516 [details]
A sample debug options file
Comment 11 Ricky Ng-Adam CLA 2006-10-23 12:09:15 EDT
Created attachment 52520 [details]
Output I get when running Eclipse / Clean Up... as instructed
Comment 12 Maxime Daniel CLA 2006-10-24 03:42:22 EDT
Analysis: a call to SourceTypeBinding#storedAnnotations(true) happens after a cleanup() has been called upon the said source type binding (in the normal course of a reconcile operation). In such case, its scope is null and should not be accessed - hence we could simply protect against this. We should still make a decision regarding whether or not annotations should be initialized in such case, given the fact that it is currently subject to a test on the scope. My take would be that it's too late to add sensible annotations to the annotations store at that point in the binding lifecycle, hence we should return null.
Will implement a patch along those lines and get it reviewed.
Comment 13 Maxime Daniel CLA 2006-10-24 05:03:49 EDT
Discussed the matter with Philippe, who says we should be a little bit subtler than that.
Would it be possible for you to share the code on which the problems happen? We'd like to nail down a reproduceable test case on our side and to assert that the scope null condition is a real indicator that the annotations are null as well.
Comment 14 Ricky Ng-Adam CLA 2006-10-24 16:44:05 EDT
I have obtained permission to send you a zip of part of our sources directly to your email address with the understanding that it will be kept confidential and that you will destroy the source copies once you've reproduced the test case. 

I'm trying to cut down on what I'll be sending you as the archive is around 3MB right now.
Comment 15 Ricky Ng-Adam CLA 2006-10-24 17:11:33 EDT
Sent email "Subject: Bugzilla Bug #159631: source code"
Comment 16 Maxime Daniel CLA 2006-10-25 02:35:18 EDT
Got it, thanks a lot. Will investigate today.
Comment 17 Maxime Daniel CLA 2006-10-25 03:34:50 EDT
Reproduced with 3.2.1.
Comment 18 Maxime Daniel CLA 2006-10-25 09:04:01 EDT
Released (inactive) BatchASTCreationTests#73 in HEAD that shows the problem in a matter of a few classes, thanks again for sharing the test case. I'll destroy all copies of it on my disk.
I will check with Philippe the best way to tackle the bug, but I'd consider that creating a method binding from another one into a given SourceTypeBinding when no more scope is available on the said SourceTypeBinding makes the attempt to store annotations superfluous. Hence protecting against scope being null should be adequate, in effect having the same effect as if the compiler had the storeAnnotations unset when resolving the SourceTypeBinding in the first place. 
Comment 19 Maxime Daniel CLA 2006-10-25 10:05:24 EDT
Created attachment 52666 [details]
Suggested fix - HEAD based
Comment 20 Maxime Daniel CLA 2006-10-26 02:39:54 EDT
Tests passed.
Comment 21 Philipe Mulet CLA 2006-10-30 05:50:19 EST
Released for 3.3M3.

Maxime - pls backport for 3.2.2.
Comment 22 Frederic Fusier CLA 2006-10-31 03:50:23 EST
(In reply to comment #21)
> Released for 3.3M3.
> 
=> FIXED
Comment 23 David Audel CLA 2006-10-31 04:09:27 EST
Verified for 3.3 M3 using build I20061030-1704
Comment 24 Maxime Daniel CLA 2006-11-07 05:32:25 EST
Released for 3.2.2.
(Same patch.)
Comment 25 Maxime Daniel CLA 2006-11-07 05:48:14 EST
(In reply to comment #15)
> Sent email "Subject: Bugzilla Bug #159631: source code"
> 

All copies destroyed here. Thx again for sharing.
Comment 26 David Audel CLA 2007-01-16 05:25:04 EST
Verified for 3.2.2 using build M20070112-1200