[pde-dev] [apitools] representing API

[Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>]

Olivier, Darin and I were talking on
Friday about how to represent the API for a given class/package/bundle/jar/...
 Our first cut was to define some set of information that is needed
and then write that to a file in some form.  Pressing on this a bit
more (i.e., looking at what would be needed) it started to look pretty
complex.  That is, to support all the queries etc we want to do, quite
a rich set of data is needed.  That in turn would imply some relatively
complex infrastructure to manage the data and the queries.

Then we revisited something Darin had
suggested a while ago, why not just use the classfiles themselves?  Initially
this approach was rejected because of a) the bulk and b) the issues people
will have with publishing their actual binaries just to support API usage
scanning.  In reconsidering this we hit upon the idea of essentially
stripping all the irrelevent (for our purposes) information from the class
files and producing "API stub" class files that contain, in their
full-featured form, 
1) API signatures
2) all external references (where external
is relative to some packaging notion defined by the context.  e.g.,
bundles in the OSGi context)
3) usage information (e.g., is a reference
a result of a field reference, method call, class ref, ...)
4) line number information (where the
reference is occurs)
5) supertype and interface information

Users should be able to optionally exclude
each of these pieces of information to suit their needs.

These API stub classes would then be
packaged together in a form similar to that of the original class files
(e.g., as bundles) for the distribution or use of the API tooling (and
other stuff.  see below).  For example, API stub bundles would
contain manifest information similar to that of the original.

Packagings (or potentially the API stub
classes themselves) would also contian supplementary API information such
as whether or not clients can call certain methods, extend or implement
various types, etc.

Cool things this all implies
- created, distributed and managed using
the same provisioning/build infrastructure as the rest of Eclipse.
- Potential to compile against these
stubs (this is useful when trying to validate lower-bound version range
requirements)
- The API stub classes could be managed
using all the regular JDT infrastructure
- Many of the existing PDE concepts
and infrastructure (e.g., Target platforms) could be used to manage API
stub lineups (e.g., API baselines)
- by maintaining the dependency info
etc we can allow the same API analysis info (i.e., the stubs) to be reused
in various configurations that match the runtime configuration.  otherwise
people would have to maintain whole "configuration snapshots"
and all the resultant duplicate information.
- doing the analysis/management work
on at a finer grain paves the way towards incremental API tooling

Questions:
- The first thing we need to understand
is the sizing of this.  How big are the stub classes in the variations
described above.  For example, if you just want to do API compatibility
testing and compilation you only need signatures (#1) and hierarchy (#5)
- what does pack200 do with these classes?
 Is there much compression to be had?
- how will we corelate orignal binary,
source and API stub packagings (e.g., it would be great if the API stubs
could be transparently substituted in for the real thing for compilation/build).

Anyway, this note is just to expose
the idea and see if anyone has further thoughts.  I believe that Olivier
is going to prototype the stub creation approach and start to answer some
of these questions.

Jeff