Bug 402393 - TagScanner and markers treat package default members as API
Summary: TagScanner and markers treat package default members as API
Status: RESOLVED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: API Tools (show other bugs)
Version: 4.3   Edit
Hardware: PC Windows 7
: P3 major (vote)
Target Milestone: 4.3 M7   Edit
Assignee: Michael Rennie CLA
QA Contact:
URL:
Whiteboard:
Keywords: readme
: 404329 405172 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-03-04 17:41 EST by Curtis Windatt CLA
Modified: 2013-05-28 07:54 EDT (History)
4 users (show)

See Also:


Attachments
fix (8.93 KB, patch)
2013-04-09 12:13 EDT, Michael Rennie CLA
no flags Details | Diff
TagScanner fix (33.83 KB, application/octet-stream)
2013-04-09 12:16 EDT, Michael Rennie CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Curtis Windatt CLA 2013-03-04 17:41:38 EST
In an API type or interface, add a method or field with a package default scope.  The javadoc quickfixes will allow you to add a @noreference restriction.  The TagScanner will inspect the tags and they will be added to the .api_description file.

This isn't correct as a package default level API could only be accessed by another API class in the same package.  Since the restrictions are intended for clients, not for the bundle's internal use, there is not value in allowing the restrictions.
Comment 1 Curtis Windatt CLA 2013-03-04 17:44:19 EST
Mike, what are your thoughts on this?  It looks like package default methods are generally treated as valid API in the API tools code.

The fix for the tag scanner is to modify org.eclipse.pde.api.tools.internal.provisional.scanner.TagScanner.Visitor.isPrivate(int)

The quick fixes and bad tag markers will require additional work.
Comment 2 Michael Rennie CLA 2013-03-05 10:16:26 EST
(In reply to comment #1)
> Mike, what are your thoughts on this?  It looks like package default methods
> are generally treated as valid API in the API tools code.
> 

This is wrong, a default scope member cannot be accessed by another bundle - the case that would cause an API error to be created.

It looks like the fix to bug 253242 was only half complete - we should add an additional check in like Flags.isPackageDefault(..) to the isPrivate method you mentioned.

> The quick fixes and bad tag markers will require additional work.

These are also wrong and should be fixed - since they would no longer be scanned + added to the API description (with the fix above) allowing them to have tags at all would be misleading. We should still allow the 'bad tag scanner' to find tags on them to report that they are not allowed.
Comment 3 Michael Rennie CLA 2013-03-25 12:35:59 EDT
Pushed fix to: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=d4d7ad36cdea388cee28d5a456cd21680f1578d2

There are still a couple of test failures to sort out.
Comment 4 Curtis Windatt CLA 2013-03-25 21:00:27 EDT
http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=02e544def114a3f9529e865f1f2be6c7fe1c26f2

Fixed the failing tests
Added a ton more tag tests covering all the valid builder messages
Fixed a couple minor problems
Added quotes to the test launch config as my workspace path has a space in it

I had a couple intermittent failures while running the tests.  I couldn't reproduce them and they weren't related to any of the changes here.
Comment 5 Curtis Windatt CLA 2013-03-26 10:51:45 EDT
*** Bug 404329 has been marked as a duplicate of this bug. ***
Comment 6 Dani Megert CLA 2013-03-27 04:07:31 EDT
TagScannerTests still fail in N20130326-2000.
Comment 7 Curtis Windatt CLA 2013-03-27 17:41:33 EDT
http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=d38693f9e1afc34e351f94df2d11357cde21f7ba
Fixes the TagScannerTests

Leaving this open to discuss with Mike the behaviour of IApiDescription.resolverAnnotations().  The API says it will return the parent package annotation if the element isn't part of the description.  However, it ends up returning the restrictions on the parent element in the following case:

public class A {
  @noreference
  public int x;
}
class B {
}

If I request annotations on class B I get the class A visibility (API).
If I remove the @noreference, requesting annotations on class B returns null.

The inconsistent behaviour in the tests is because of pruning in the TagScanner.  If there are no restrictions the description's package map is never created.
Comment 8 Dani Megert CLA 2013-03-28 03:49:03 EDT
Tests are green now (N20130327-2100).
Comment 9 Michael Rennie CLA 2013-04-01 15:49:05 EDT
(In reply to comment #7)
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=d38693f9e1afc34e351f94df2d11357cde21f7ba
> Fixes the TagScannerTests
> 
> Leaving this open to discuss with Mike the behaviour of
> IApiDescription.resolverAnnotations().  The API says it will return the
> parent package annotation if the element isn't part of the description. 
> However, it ends up returning the restrictions on the parent element in the
> following case:
> 

The doc could use some clean-up. It will only return the parent description if the parent has one. Using the example:

/**
*
*/
public class A {
  /**
  * @noinstantiate
  */
  class B {
  }
}

null is returned (and expected) when asking about B - (1) because B does not appear in the description now, and (2) A is unrestricted and we have never added unrestricted parent nodes to the description if none of their children are restricted.

> The inconsistent behaviour in the tests is because of pruning in the
> TagScanner.  If there are no restrictions the description's package map is
> never created.

This is sort of expected, both cases mean unrestricted (null and otherwise), but in the null case the parent is not even added to the description as mentioned above.

I will fix the commented out tests and update the doc on IApiDescripton.
Comment 10 Michael Rennie CLA 2013-04-01 17:05:13 EDT
(In reply to comment #9)

> I will fix the commented out tests and update the doc on IApiDescripton.

http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=ee96ffb50dbd979f69079f3be4ccca4cf4a1ddc8

I found an issue with our invalid tag scanner, it appears it does not thoroughly check the visibility of the a member. Consider the following example:

public class PublicStaticInner {
  static class Inner {
    /**
     * @noinstantiate This class is not intended to be instantiated by clients.
     * @noextend This class is not intended to be subclassed by clients.
     */
    public static class OpenInner {
      /**
	* @noreference This method is not intended to be referenced by clients.
	*/
	public static void one() {
	  System.out.println("one");
	}
    }
  }
}

Those tags are not flagged as invalid when they should be.
Comment 11 Curtis Windatt CLA 2013-04-02 13:29:42 EDT
I've created a separate bug (Bug 404744) for the problem Mike found as it is independent of this change and needs to be fixed for private types as well as package default.

Marking this bug as FIXED.
Comment 12 Dani Megert CLA 2013-04-05 09:00:07 EDT
This causes two issues:

1. I now get an unexpected "missing @since tag" API error on a field which has
   such an unsupported warning.

2. The warning I get on IJavaColorConstants.PREFIX (jdt.ui) is gibberish:
    Tag '@noreference' is unsupported on a package default field

    The buildermessages.properties file has several similar entries, which also 
    need work.


Marking as 'major', since I now have errors in my workspace.
Comment 13 Markus Keller CLA 2013-04-05 09:26:40 EDT
The new warning message
"Tag '@noreference' is unsupported on a package default field"
is wrong for interface fields.

Example:

public interface IJavaColorConstants {
	/**
	 * @noreference This field is not intended to be referenced by clients.
	 */
	String PREFIX= "java_"; //$NON-NLS-1$


The field PREFIX is *not* package default. It is implicitly public, static,
and final, see JLS7 9.3 or the Eclipse Outline view. The message should be:
"Tag '@noreference' is unsupported on a final field".


(In reply to comment #12)
I think we have to live with these problems, since a classfile-based API Tools implementation cannot support @noreference on constant fields. Workaround is to remove @noreference and add an API filter for the missing @since tag for now. The filter can be removed as soon as the API baseline profile has been regenerated and contains the previously-excluded field.
Comment 14 Dani Megert CLA 2013-04-05 12:08:00 EDT
> (In reply to comment #12)
> I think we have to live with these problems, since a classfile-based API
> Tools implementation cannot support @noreference on constant fields.
> Workaround is to remove @noreference and add an API filter for the missing
> @since tag for now. The filter can be removed as soon as the API baseline
> profile has been regenerated and contains the previously-excluded field.

I recompiled my big source workspace, and the reported issue is the only one. Hence, I'm fine with adding a filter (did that for JDT). BUT: we need to add this to the migration guide and README, so that people know, what's behind this unexpected error.
Comment 15 Michael Rennie CLA 2013-04-09 09:33:19 EDT
*** Bug 405172 has been marked as a duplicate of this bug. ***
Comment 16 Michael Rennie CLA 2013-04-09 12:13:13 EDT
Created attachment 229516 [details]
fix

fix that still has some test failures due to changed messages.
Comment 17 Michael Rennie CLA 2013-04-09 12:16:03 EDT
Created attachment 229517 [details]
TagScanner fix

The other patch has a patch for our completion proposal computer and this patch is for the tag scanner
Comment 18 Michael Rennie CLA 2013-04-10 15:47:14 EDT
pushed fix + test updates to:

http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=391927e196dd319d6ba19497af094dde9d4804d3

I am going to leave this open until I have a build to test to make sure all is well.
Comment 19 Dani Megert CLA 2013-04-12 02:44:14 EDT
(In reply to comment #18)
> pushed fix + test updates to:
> 
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=391927e196dd319d6ba19497af094dde9d4804d3
> 
> I am going to leave this open until I have a build to test to make sure all
> is well.

7 failures in N20130411-2000.

http://download.eclipse.org/eclipse/downloads/drops4/N20130411-2000/testresults/html/org.eclipse.pde.api.tools.tests_macosx.cocoa.x86_5.0.html
Comment 20 Michael Rennie CLA 2013-04-12 12:12:06 EDT
(In reply to comment #19)
> (In reply to comment #18)
> > pushed fix + test updates to:
> > 
> > http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> > ?id=391927e196dd319d6ba19497af094dde9d4804d3
> > 
> > I am going to leave this open until I have a build to test to make sure all
> > is well.
> 
> 7 failures in N20130411-2000.
> 
> http://download.eclipse.org/eclipse/downloads/drops4/N20130411-2000/
> testresults/html/org.eclipse.pde.api.tools.tests_macosx.cocoa.x86_5.0.html

Fixed in:

http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=d7fd9fd18ad35c0c2044cfe8756758724e2af3ee

The commit also fixes the case for the nooverride tag on static methods
Comment 21 David Williams CLA 2013-04-12 14:05:14 EDT
(In reply to comment #20)
> (In reply to comment #19)
> > (In reply to comment #18)
> > > pushed fix + test updates to:
> > > 
> > > http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> > > ?id=391927e196dd319d6ba19497af094dde9d4804d3
> > > 
> > > I am going to leave this open until I have a build to test to make sure all
> > > is well.
> > 
> > 7 failures in N20130411-2000.
> > 
> > http://download.eclipse.org/eclipse/downloads/drops4/N20130411-2000/
> > testresults/html/org.eclipse.pde.api.tools.tests_macosx.cocoa.x86_5.0.html
> 
> Fixed in:
> 
> http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/
> ?id=d7fd9fd18ad35c0c2044cfe8756758724e2af3ee
> 
> The commit also fixes the case for the nooverride tag on static methods

org.eclipse.wst.jsdt.core.dom ... seriously? You know that's not "one of ours", right? :) 

Not sure if you committed that by accident, or really want to use it ... but, causes build to fail pretty quick (since it "can't be found").
Comment 22 Michael Rennie CLA 2013-04-12 14:33:01 EDT
(In reply to comment #21)
> Not sure if you committed that by accident, or really want to use it ...
> but, causes build to fail pretty quick (since it "can't be found").

I'm not even sure how that got in there. But it has been removed.
Comment 23 Dani Megert CLA 2013-04-15 03:56:16 EDT
Tests pass now.
Comment 24 Michael Rennie CLA 2013-04-17 09:57:44 EDT
Marking fixed, all the tests pass and smoke testing has not revealed anything new.
Comment 25 Dani Megert CLA 2013-05-28 07:54:00 EDT
Filed bug 409271 for the readme entry. Looks like PDE doc does not have a migration guide section.