Bug 510381 - API Tools report and IDE disagree
Summary: API Tools report and IDE disagree
Status: VERIFIED FIXED
Alias: None
Product: PDE
Classification: Eclipse Project
Component: API Tools (show other bugs)
Version: 4.7   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: 4.7 M6   Edit
Assignee: Vikas Chandra CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on:
Blocks:
 
Reported: 2017-01-12 11:26 EST by Dani Megert CLA
Modified: 2017-03-06 07:32 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dani Megert CLA 2017-01-12 11:26:25 EST
I20170112-0245.

API Tools report does not report all problems.

For org.eclipse.osgi.services it only reports one compatibility warning:

http://download.eclipse.org/eclipse/downloads/drops4/I20170112-0245/apitools/analysis/html/org.eclipse.osgi.services/report.html

However, in the IDE it gives 19 errors:
1. install latest I-build
2. start with a new workspace
3. set API Tools baseline to 4.6.2
4. clone C:\eclipse\workspaces\HEAD\Git\rt.equinox.framework\.git
5. checkout tag I20170112-0245
6. import org.eclipse.osgi.services

==> 19 (expected) errors on org.eclipse.osgi.services
Comment 1 Dani Megert CLA 2017-01-12 12:22:26 EST
Bug 434799 might be related.

There are various mysteries here:

1. The following error is reported in the report but not in the IDE:
  - The field org.osgi.service.component.annotations.Component.NAME in an 
    annotation that is intended to be extended has been addedis reported in the
    report but not in the IDE

2. The following errors are reported in the IDE but not in the report
  - The method org.osgi.service.component.annotations.Component.reference() 
    without a default value has been added
  - The methodorg.osgi.service.component.annotations.Component.configurationPid()
    with a default value has been removed

3. Invalid @since tag errors are not in the report


So, the first task is to figure out what the correct behavior for 1. and 2. is and then fix the report and/or the IDE.

For 3. we need to know who excludes them. It can be the report generator itself or maybe a command line argument during the build.


NOTE: The project might contain wrong/invalid API Tools filters (probably not against the 4.6.2 baseline).
Comment 2 Vikas Chandra CLA 2017-01-18 10:30:14 EST
The link http://download.eclipse.org/eclipse/downloads/drops4/I20170112-0245/apitools/analysis/html/org.eclipse.osgi.services/report.html

has

Compatibility
The field org.osgi.service.component.annotations.Component.NAME in an annotation that is intended to be extended has been added


- updating this since the link may become non-functional
Comment 3 Vikas Chandra CLA 2017-01-18 11:01:34 EST
I am able to recreate comment#0.

The prime difference is that the IDE is giving an "invalid since" and "missing since errors" with respect to version of the plugin whereas the tool report takes into account exported package versioning and considers that version as the appropriate version for since addition versioning. 

Having said that, the 1st step is find out which one is correct behavior and ensure consistency. Also I need to investigate issues that "may" not be related to this but present in comment#0 and comment#1.

I am investigating this on a high priority.
Comment 4 Vikas Chandra CLA 2017-01-18 11:50:43 EST
Regarding 3) of comment#1,

I20170112-0245 has the since tag related issues. The latest master in IDE doesn't have because they have been filtered in the project. So it is more or less certain that the tag related issues are correct API tool behaviour and they have been also excluded in "I20170112-0245/apitools/analysis/html/org.eclipse.osgi.services/report.html" ( yet to investigate how but this should be straightforward ). 

This still leaves us with 3 issues ( started with same number of items)
1)  Error is reported in the report but not in the IDE 
2)  Errors are reported in the IDE but not in the report
3)  How is the "invalid since" filtered out on API tool report
Comment 5 Dani Megert CLA 2017-01-19 05:15:25 EST
Note that after Markus cleaned up the API filters, we now get an additional warning in the latest report:

The major version should be incremented in version 3.6.0, since API breakage occurred since version 3.5.100

This fits with the original/other warning.


The difference between the report and the IDE is that the report uses API descriptions for both builds, while in the IDE the API descriptions are only used for the baseline.

Might be related to bug 496145.


NOTE: Sravan double-checked that the baseline in the build is correct (4.6.2).
Comment 6 Vikas Chandra CLA 2017-01-31 04:24:24 EST
APIToolsAnalysisTask doesnt process category case IApiProblem.CATEGORY_SINCETAGS: 
( see line no 68 in APIToolsAnalysisTask.java).

The major version increase problem is most likely a side-effect of  bug 496145.

>The difference between the report and the IDE is that the report uses API >descriptions for both builds, while in the IDE the API descriptions are only >used for the baseline.

As of now, I am not sure if they account for the other 2 inconsistencies.

I ran the apitooling.analysis & apitooling.analysis_reportconversion locally ("without filters")

and I am getting these errors.

The field org.osgi.service.component.ComponentConstants.COMPONENT_CAPABILITY_NAME in an interface that is intended to be implemented or extended has been added
The field org.osgi.service.component.annotations.Component.NAME in an annotation that is intended to be extended has been added
The method org.osgi.service.component.annotations.Component.configurationPid() with a default value has been removed
The method org.osgi.service.component.annotations.Component.reference() without a default value has been added

I am investigating further on this.
Comment 7 Vikas Chandra CLA 2017-01-31 08:20:48 EST
>There are various mysteries here:

>1. The following error is reported in the report but not in the IDE:
>  - The field org.osgi.service.component.annotations.Component.NAME in an 
>    annotation that is intended to be extended has been addedis reported in the
>    report but not in the IDE

This is because the problem is filtered in  the IDE ( wrongly). The preference KEY calculated is ANNOTATION_ELEMENT_TYPE_ADDED_FIELD which doesn't exist in API tools at all. ( see line 30 in IApiProblemTypes).

So the code 
	String value = service.get(prefkey, null, getPreferences(context));
		if (VALUE_ERROR.equals(value)) {
			return SEVERITY_ERROR;
		}
		if (VALUE_WARNING.equals(value)) {
			return SEVERITY_WARNING;
		}
		return SEVERITY_IGNORE;

returns ignore and that API problem is ignored ( always if the component is there in the workspace).

However if that component is not there ( for example the nightly build API tools task)
private boolean isProblemFiltered(IApiProblem problem) {
		if (fJavaProject == null) {
			if (this.fFilterStore != null) {
                         //something
	                 }
			// something more.. etc
			}
			return false;

if the filter store is null, isProblemFiltered can return false which shows this error.

If this type of API tool problem is something that is not an issue, it should be never created in the 1st place but I think the correct thing here is to have a preference key for  ANNOTATION_ELEMENT_TYPE_ADDED_FIELD and make IDE consistent with the report page.

As far as I can understand, probably this never worked in IDE API tool view.
Comment 8 Vikas Chandra CLA 2017-01-31 08:49:51 EST
>2. The following errors are reported in the IDE but not in the report
>  - The method org.osgi.service.component.annotations.Component.reference() 
>    without a default value has been added
>  - The methodorg.osgi.service.component.annotations.Component.configurationPid()
>    with a default value has been removed


In I20170112-0245, the following filter is used in workspace. Here the resource path is wrong

    <resource path="META-INF/MANIFEST.MF" type="org.osgi.service.component.annotations.Component">
        <filter id="286375965">
            <message_arguments>
                <message_argument value="org.osgi.service.component.annotations.Component"/>
                <message_argument value="reference()"/>
            </message_arguments>
        </filter>

The correct filter id should be similar to  ( see the resource path)

  <resource path="src/org/osgi/service/component/annotations/Component.java" type="org.osgi.service.component.annotations.Component">
        <filter id="288469092">
            <message_arguments>
                <message_argument value="org.osgi.service.component.annotations.Component"/>


I have unable to retrieve the api filters here - http://download.eclipse.org/eclipse/downloads/drops4/I20170112-0245/testResults.php but I suspect they may be correct as in the 26th build and hence these errors didn't appear in build page. 

The only pending items on this bug ( see previous comment) is adding a preference key for  ANNOTATION_ELEMENT_TYPE_ADDED_FIELD and make IDE consistent with report.
Comment 9 Eclipse Genie CLA 2017-01-31 11:53:41 EST
New Gerrit change created: https://git.eclipse.org/r/89988
Comment 10 Vikas Chandra CLA 2017-01-31 12:17:37 EST
Test case is already there AnnotationDeltaTests::test9. However the test collects the deltas and checks the type pf delta. It doesn't create an API problem and check if problem is filtered etc. Hence those test always passes.
Comment 11 Vikas Chandra CLA 2017-02-08 08:53:27 EST
I used then changes from 89988 to launch eclipse and test with a subset of eclipse SDK code . I got only 1 extra error which is
 
The field org.osgi.service.component.annotations.Component.NAME in an annotation that is intended to be extended has been added

which is already present in http://download.eclipse.org/eclipse/downloads/drops4/I20170207-2000/apitools/analysis/html/org.eclipse.osgi.services/report.html

The only thing that happens now ( with this fix)  is that it shows up in IDE also ( instead of just API report - inconsistent) and is now controlled by a preference.
Comment 13 Vikas Chandra CLA 2017-02-09 03:47:56 EST
Thanks Dani for spotting these glaring inconsistencies and helping with the review and also of analysis of since tag inconsistency.
Comment 14 Vikas Chandra CLA 2017-02-09 03:48:43 EST
I am marking this as noteworthy since a new UI option has been added.
Comment 15 Vikas Chandra CLA 2017-03-06 07:32:04 EST
verified comment#11 with

Version: Oxygen (4.7)
Build id: I20170222-2000