RE: [jdt-core-dev] Pattern-Based Formatter Prototype

["Dave Wathen" <dave.wathen@xxxxxxxxxxxx>]

Hi Olivier,

Good to hear from you again.

>1) You don't talk too much about the selection of a rule rather than
>another one. For example, the distinction between a casting and the
>arguments of a method.

Right, this is one of the areas on which the prototype is quite light.
There are basically two mechanisms:

The first aspect is context.  When the instructions of a matched rule
include a process instruction then a sub-section of the matched tokens
have rules applied to them.  The rules applied in this sub-processing
can be limited to a given ruleset.  Think of it as very similar to a
recursive decent parser.

The second aspect is to distinguish patterns that are evaluated in
a single context.  In general I assume the context will reduce such
conflicts (just as it does in the parser). For those conflicts that
remain there are many approaches available:

- Greediness - that is the pattern capable of eating the most tokens
  should win.

- Specifity - that is a pattern to match (say) a modifier is more
  specific than a token to match any.

- Rule priority - This was the only aspect implemented for the prototype
  and is denoted by the order in which the rules are given.  For a proper
  implementation this should probably be the order in the ruleset though.

The two remaining risks that I can see are:

a) Reduced opportunity for context when only a part of the source is to
be formatted - unless we process the whole source anyway but only format
the requested section.  Presumably the parser approach suffers this too.

b) The token stream doesn't distinguish a identifier between a type and
a non-type.  Heuristics can almost certainly be defined to (at least)
approximate this.

>2) How do you think it is possible to add the mapping between positions
>before formatting and positions after formatting?

I imagine this to be fairly straight-forward. (I just didn't think putting
it in the prototype proved anything.)  ITokenArray defines that the start
and end position of each token from the original stream are available.

As a write instruction causes a token to be output this information
should be passed to Scribe which can then provide the mapping function.
Scribe will need to buffer the mappings (as it does with other data)
to continue to support the transactional nature.

At the end of formatting the mappings can be gathered from Scribe and used
to update markers (and anything else required).

>3) If I well understood, only the Cfss parts depend on ANTLR. For licencing
>issues, we might want to get rid of this dependency.

Correct.  ANTLR is open source but the licence isn't any of the usual
suspects (in fact I don't think there's even a copyright on it).  ANTLR
could be eliminated - it was just quick and easy to use for the prototype.

>4) How confident are you to be able to address the improvements listed
>above with this approach?

There seem to be two parts to this question; how confident am I that the
approach can address these issues and am I able to do so.

As to the approach's applicability to these issues I'm comfortable that a
full
implementation of the approach can address these issues.  I've looked at the
quoted bug reports.  The one area that would need lots more attention than
it got in the prototype is handling of existing whitespace.  I'd anticipated
this and commented on it in the document accompanying the prototype (see
Observations - second point in missing functionality and last point in
general
observations).  Some issues, such as variable alignment and end-of-line
comment alignment can be addressed by adding features to Scribe.

As to the involvement in this I can offer it's very awkward for me to say
at this point in time.  I've been completely unable to find any contract
work and I'm now looking for a permanent position.  Without knowing what
the future holds it's difficult for me to commit.  I'd very much like to
see this idea taken to fruition and I'm prepared to give what time I can
but that may be very little (especially if, as seems likely, I go back to
to the world of investment banking).  I'm sure I could at a minimum act
as a sounding board to see the design through.

So perhaps the more important question is how confident are YOU that you
can extrapolate the prototype to a full implementation and are you
interested in doing so?

Dave Wathen
Canzonet Limited
Phone: +44 (0)20 8660 5171
Mobile: +44 (0)7968 167934
Fax: +44 (0)870 051 7664
http://www.canzonet.com
mailto:dave.wathen@xxxxxxxxxxxx


_______________________________________________
jdt-core-dev mailing list
jdt-core-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/jdt-core-dev