Bug 258906 - [jsr269] Package annotations not visible to Java 6 APT processors
Summary: [jsr269] Package annotations not visible to Java 6 APT processors
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.4.1   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 3.5 M5   Edit
Assignee: Kent Johnson CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 214948
Blocks:
  Show dependency tree
 
Reported: 2008-12-16 04:40 EST by Vladimir Piskarev CLA
Modified: 2009-01-27 06:51 EST (History)
3 users (show)

See Also:


Attachments
'testprocessor' plug-in project (3.29 KB, application/zip)
2008-12-16 04:40 EST, Vladimir Piskarev CLA
no flags Details
'test' project (2.04 KB, application/zip)
2008-12-16 04:41 EST, Vladimir Piskarev CLA
no flags Details
Test case (4.87 KB, patch)
2008-12-19 08:52 EST, Vladimir Piskarev CLA
no flags Details | Diff
Candidate patch (2.30 KB, patch)
2008-12-30 07:41 EST, Srikanth Sankaran CLA
no flags Details | Diff
Revised patch with test (8.01 KB, patch)
2008-12-31 08:53 EST, Srikanth Sankaran CLA
no flags Details | Diff
Proposed patch (3.23 KB, patch)
2009-01-05 12:49 EST, Kent Johnson CLA
no flags Details | Diff
Proposed patch and testcase (8.87 KB, patch)
2009-01-07 11:29 EST, Kent Johnson CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Piskarev CLA 2008-12-16 04:40:04 EST
Build ID: M20080911-1700

Steps To Reproduce:
1. Import the soon-to-be-attached 'testprocessor' plug-in project into your workspace.
2. From your workspace, run a PDE workspace.
3. Import the soon-to-be-attached 'test' project into the PDE workspace.
4. Note the build error 'XML namespace must be given in @XmlType or @XmlSchema'.


More information:
Really, this build error should not occur because XML namespace _is_ given in @XmlSchema annotation (see 'package-info.java'). The problem is caused by package annotations not being visible to Java 6 annotation processors. Namely, calls to PackageElement#getAnnotationMirrors seem to always return empty list.
Comment 1 Vladimir Piskarev CLA 2008-12-16 04:40:58 EST
Created attachment 120549 [details]
'testprocessor' plug-in project
Comment 2 Vladimir Piskarev CLA 2008-12-16 04:41:30 EST
Created attachment 120550 [details]
'test' project
Comment 3 Walter Harley CLA 2008-12-16 12:56:23 EST
Looks like the same underlying cause as bug 214948 - in a nutshell, package annotations don't get along with incremental compilation.  Not yet clear what the solution should be.
Comment 4 Kent Johnson CLA 2008-12-16 15:29:41 EST
Walter - do you use the DOM's PackageBinding to get the annotations or the compiler's PackageBinding?

Annotations have never been available on the compiler's PackageBinding, but they are available on the DOM's.
Comment 5 Kent Johnson CLA 2008-12-16 16:56:26 EST
Ok - so I found PackageElementImpl.getAnnotationBindings()

We're trying to get the annotations on the package-info source type binding but they aren't there.

I'll keep looking.
Comment 6 Kent Johnson CLA 2008-12-17 16:35:08 EST
Moving to JDT Core since the annotations were not attached to the package properly.
Comment 7 Vladimir Piskarev CLA 2008-12-19 08:52:38 EST
Created attachment 120937 [details]
Test case

Patch to 'org.eclipse.jdt.compiler.apt.tests' with a test case for the issue.
Comment 8 Srikanth Sankaran CLA 2008-12-30 00:52:10 EST
(In reply to comment #7)
> Created an attachment (id=120937) [details]
> Test case
> Patch to 'org.eclipse.jdt.compiler.apt.tests' with a test case for the issue.

Hello Vladimir, I am going to need some instructions on how to run these tests properly. If I checkout the project 'org.eclipse.jdt.compiler.apt.tests', apply the patch in comment #7, then run the tests (TestAll) agains HEAD versions of JDT core projects, I get a couple of null pointer exceptions as shown below. Let me know what additional steps are needed for proper executiuon of this test.
(I am using 3.5 M4)

junit.framework.AssertionFailedError: Compilation failed : Round 1:
	input files: {targets.model.pb.AB,targets.model.pa.A,targets.model.pb.AC,targets.model.pb.D,targets.model.pc.F,targets.model.pc.G,targets.model.pc.H,targets.model.pc.J,targets.model.pc.Deprecation,targets.model.pc.AnnotatedWithManyTypes,targets.model.pa.ExceptionA,targets.model.pa.AnnoZ,targets.model.pc.AnnoY,targets.model.pc.AnnoX,targets.model.pb.DChild,targets.model.pa.IA,targets.model.pb.IB,targets.model.pb.IC,targets.model.pc.Overriding,targets.model.pc.IF}
	annotations: [org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoArrayType,java.lang.Deprecated,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoArrayInt,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoString,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoArrayAnnoChar,java.lang.annotation.Inherited,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoAnnoChar,targets.model.pa.AnnoZ,targets.model.pc.AnnoX,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoArrayEnumConst,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoInt,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoFloat,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoLong,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoArrayString,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoDouble,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoType,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoByte,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoChar,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoBoolean,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoShort,java.lang.Override,targets.model.pc.AnnoY,java.lang.SuppressWarnings,org.eclipse.jdt.compiler.apt.tests.annotations.TypedAnnos.AnnoEnumConst]
	last round: false
----------
1. ERROR in C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\eclipse-temp\src\targets\model\pc\Overriding.java (at line 0)
	/*******************************************************************************
	^
Internal compiler error: java.lang.NullPointerException at org.eclipse.jdt.internal.compiler.apt.model.TypesImpl.getDeclaredType(TypesImpl.java:258)
----------
1 problem (1 error)
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.Assert.assertTrue(Assert.java:20)
	at org.eclipse.jdt.compiler.apt.tests.BatchTestUtils.compileTree(BatchTestUtils.java:121)
	at org.eclipse.jdt.compiler.apt.tests.ModelUtilTests.internalTest(ModelUtilTests.java:88)
	at org.eclipse.jdt.compiler.apt.tests.ModelUtilTests.testTypesWithEclipseCompiler(ModelUtilTests.java:72)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:45)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:599)
	at junit.framework.TestCase.runTest(TestCase.java:164)
	at junit.framework.TestCase.runBare(TestCase.java:130)
	at junit.framework.TestResult$1.protect(TestResult.java:106)
	at junit.framework.TestResult.runProtected(TestResult.java:124)
	at junit.framework.TestResult.run(TestResult.java:109)
	at junit.framework.TestCase.run(TestCase.java:120)
	at junit.framework.TestSuite.runTest(TestSuite.java:230)
	at junit.framework.TestSuite.run(TestSuite.java:225)
	at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
	at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62)
	at org.eclipse.pde.internal.junit.runtime.CoreTestApplication.run(CoreTestApplication.java:23)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:45)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:599)
	at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:195)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:366)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:45)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:599)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:550)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:505)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1237)
	at org.eclipse.equinox.launcher.Main.main(Main.java:1213)



Comment 9 Srikanth Sankaran CLA 2008-12-30 01:02:19 EST
(In reply to comment #8)
> (In reply to comment #7)
> > Created an attachment (id=120937) [details] [details]
> > Test case
> > Patch to 'org.eclipse.jdt.compiler.apt.tests' with a test case for the issue.
> Hello Vladimir, I am going to need some instructions on how to run these tests
> properly. If I checkout the project 'org.eclipse.jdt.compiler.apt.tests', apply
> the patch in comment #7, then run the tests (TestAll) agains HEAD versions of
> JDT core projects, I get a couple of null pointer exceptions as shown below.
> Let me know what additional steps are needed for proper executiuon of this
> test.

I also meant to mention that with or without applying the patch for the test I get these NPE related failures.
Comment 10 Vladimir Piskarev CLA 2008-12-30 01:33:19 EST
Here are the steps:

- Check out HEAD org.eclipse.jdt.compiler.apt.tests
- Apply the patch
- Right-click on apttestprocessors.jardesc and select "Create Jar"
- Right-click AllTests.java and select "Run As -> JUnit Plug-in Test"

You should get 1 failure on "testElementWithEclipseCompiler".
Comment 11 Srikanth Sankaran CLA 2008-12-30 02:42:30 EST
(In reply to comment #10)
> Here are the steps:
> - Check out HEAD org.eclipse.jdt.compiler.apt.tests
> - Apply the patch
> - Right-click on apttestprocessors.jardesc and select "Create Jar"
> - Right-click AllTests.java and select "Run As -> JUnit Plug-in Test"
> You should get 1 failure on "testElementWithEclipseCompiler".

No improvements. I still get the same NPEs. Here is some more information.
org.eclipse.jdt.compiler.apt.tests.processors.typeutils.TypeUtilsProc.examineGetDeclaredTypeNested() does a lookup 
TypeElement iterDecl = _elementUtils.getTypeElement("java.util.HashMap.HashIterator");

getTypeElement returns null. (I don't see HashIterator in JRE 6.). This null reference gets passed down to 

DeclaredType decl = _typeUtils.getDeclaredType(outerType, iterDecl, new TypeMirror[] { numberArrayType });

from inside which arises the NPE mentioned earlier. 


Should I be building/running with any specific java level compliance ?

BTW, I have a fix for the original problem of this bug and I am able to verify it against the original set of tests posted by you in comment #1 and comment #2.
Comment 12 Vladimir Piskarev CLA 2008-12-30 05:44:01 EST
Thank you for the fix, Srikanth!

I'm building and running with JDK 6  and have no such problem. Maybe you should also check out HEAD of org.eclipse.jdt.compiler.apt and org.eclipse.jdt.core. I have the following setup:

- Sun JDK 6
- Eclipse 3.4.1
- HEAD of org.eclipse.jdt.compiler.apt, org.eclipse.jdt.compiler.apt.tests, org.eclipse.jdt.core checked out in the workspace
Comment 13 Vladimir Piskarev CLA 2008-12-30 06:28:58 EST
Srikanth, the problem you're getting is probably caused by Sun vs. IBM JDK implementation differences. Sun JDK 6 does have the (private) type java.util.HashMap.HashIterator, but it is missing in IBM's one.
Comment 14 Srikanth Sankaran CLA 2008-12-30 06:40:56 EST
(In reply to comment #13)
> Srikanth, the problem you're getting is probably caused by Sun vs. IBM JDK
> implementation differences. Sun JDK 6 does have the (private) type
> java.util.HashMap.HashIterator, but it is missing in IBM's one.

Thanks, in any case the test you have created (testElementWithEclipseCompiler) fails without my fix and passes with my fix. It is the testType* that fails with or without my patch and also with or without the test patch itself.


Comment 15 Srikanth Sankaran CLA 2008-12-30 07:41:37 EST
Created attachment 121339 [details]
Candidate patch



    Kent, this is what I am considering as a possible fix. This passes both the tests attached to this bug. (The incremental vs full build problem is still there, but then that is the subject matter of the other defect (bug #214948))

    Essentially the package level annotations are now propagated to the interface type "package-info". (they are still not attached to the PackageBinding though)

    Let me know what you think. After that I'll roll up the final patch with a test case thrown in.
Comment 16 Srikanth Sankaran CLA 2008-12-31 08:53:08 EST
Created attachment 121392 [details]
Revised patch with test
Comment 17 Kent Johnson CLA 2009-01-05 12:49:04 EST
Created attachment 121550 [details]
Proposed patch

I would write the Annotation change so it matches the previous tests.

And instead of attaching the package's annotations in the parser, I would attach them in the CompilationUnitScope where we ensure the package-info type binding is created.

This is half the patch I have for bug 214948 & bug 258906, let me know when/if you want to see the rest of it.
Comment 18 Srikanth Sankaran CLA 2009-01-05 23:27:33 EST
(In reply to comment #17)
> Created an attachment (id=121550) [details]
> Proposed patch
> I would write the Annotation change so it matches the previous tests.

1. OK. I can live with that.

2. I had overlooked the change in org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.generateCode().
While strictly not needed from a correctness standpoint, what you have done is cleaner here.

> And instead of attaching the package's annotations in the parser, I would
> attach them in the CompilationUnitScope where we ensure the package-info type
> binding is created.

3. I think it is actually better to attach the annotations as early as possible i.e in the parser itself. This seems to me less ad-hoc and more structurally integral and internally consistent. See that javadoc gets propagated from CU to
package-info in the parser.

It occurs to me that change in org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.generateCode() which you have (rightly) deleted now, was in fact suffering from the same symptoms, i.e not propagating the annotations early enough.

That said, do you think the inner if from org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.buildTypeBindings(AccessRestriction) ever gets executed ?

		} else if (this.referenceContext.isPackageInfo()) {
			// resolve package annotations now if this is "package-info.java".
			if (this.referenceContext.types == null || this.referenceContext.types.length == 0) {
				this.referenceContext.types = new TypeDeclaration[1];
				TypeDeclaration declaration = new TypeDeclaration(this.referenceContext.compilationResult);
				this.referenceContext.types[0] = declaration;
				declaration.name = TypeConstants.PACKAGE_INFO_NAME;
				declaration.modifiers = ClassFileConstants.AccDefault | ClassFileConstants.AccInterface;
				firstIsSynthetic = true;
			} 

> This is half the patch I have for bug 214948 & bug 258906, let me know when/if
> you want to see the rest of it.

In summary, I am ok with either your patch or mine. As for, bug #214948, hold on for a couple more days. I have made a lot of progress there.
Comment 19 Srikanth Sankaran CLA 2009-01-06 02:15:58 EST
(In reply to comment #18)

> That said, do you think the inner if from
> org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.buildTypeBindings(AccessRestriction)
> ever gets executed ?

It does, in the case of SourceElementParser instantiated by ProblemFinder. So I guess your patch will handle that case better than mine.

Comment 20 Kent Johnson CLA 2009-01-06 10:19:27 EST
I guess the bottom line for me is: I don't trust the parser and all of its children to attach the annotations in all cases correctly (too error prone).

I know for sure that the unit scope will ensure that the package-info type exists & will attach the annotations before they're resolved.

Keep going on bug 214948, but I would like all of the annotation bugs to be closed by the end of the week.
Comment 21 Srikanth Sankaran CLA 2009-01-06 21:36:32 EST
(In reply to comment #20)
> I guess the bottom line for me is: I don't trust the parser and all of its
> children to attach the annotations in all cases correctly (too error prone).

Good point. I hadn't realized this until after I found out about the SourceElementParser. It would appear that the synthetic type cretion in the (regular) Parser is (perhaps) wasteful, but may be sleeping dogs should be left alone :)

> I know for sure that the unit scope will ensure that the package-info type
> exists & will attach the annotations before they're resolved.

OK, please proceed to close this defect with the patch you have - perhaps you can leverage the test from mine.

> Keep going on bug 214948, but I would like all of the annotation bugs to be
> closed by the end of the week.

Sounds good.

Comment 22 Kent Johnson CLA 2009-01-07 11:29:46 EST
Created attachment 121822 [details]
Proposed patch and testcase

Added Srikanth's test
Comment 23 Kent Johnson CLA 2009-01-07 12:28:50 EST
Fix and test released for 3.5M5
Comment 24 David Audel CLA 2009-01-27 06:51:01 EST
Verified for 3.5M5 using I20090126-1300