[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
[xtext-dev] Redesign of Syntax Coloring
|
Hi,
I've redesigned syntax coloring for Xtext. The goal was to use the
AntLR lexer instead of the parser for syntax coloring. I've checked in
my first attempts. The old code is still there as I haven't
implemented property pages yet.
As we rather want to use JFace interfaces than define own ones, I
chose ITokenScanner as the base service, i.e. that is the candidate
for an extension point allowing to be overridden. Behind that one, we
only allow customization using the service registry. Alternatives
would have been PresentationReconciler or DefaultDamagerRepairer, but
because we reuse the default ones here and plug in our own
ITokenScanner, I've chosen the latter.
The default implementation gets a Lexer and a ITokenColorer injected.
The AbstractAntlrTokenColorer looks up AntLR token types (integer) in
a map that is populated from the AntLR generated .tokens file. The
returned String is matched against a number of patterns
(BuiltinTokenColorer) and the appropriate TokenStyle is returned.
Note that the DefaultTokenScanner depends on AntLR API (CommonToken,
Lexer), which I don't know if it is a good idea. An alternative would
be to wrap each token in a new AntLR independent object. I like that
even less, as the main point in using the lexer instead of the parser
is to increase performance.
We should also consider getting rid of the intermediate String token
type description resulting from the .token file. Note that we already
parse this file to get lexer rule names in the parser (That's why I
extracted that in a service, even though the parser doesn't use it
yet). An alternative would be to have an AntLRTokenType (int) to
LexerRule/Keyword (AbstractElement) interface, and lookup the
TokenStyles for the latter. Hmm, that sounds much better to me now. I
should give it a try tomorrow.
Some of the code looks pretty much like it has been before, but with
different class names. This is mainly due to the fact that I did a
fresh start. It's not that I dislike what Dennis has worked out before.
I've used the service registry a lot, and I have become quite a friend
of it :-) But I see that the resulting code might be a little bit hard
to understand, as things are defined in very different places and
scopes. Maybe we can workout some best practices/guidelines on using
the service registry?
Best regards
Jan
--
Dr. Jan Köhnlein
Senior Software Architekt
Telefon: +49 (0) 431 / 5606-337
Mobile: +49 (0) 163 / 3979 776
Telefax: +49 (0) 431 / 5606-339
http://www.itemis.de
jan.koehnlein@xxxxxxxxx
itemis AG
Schauenburgerstr. 116
24118 Kiel
Rechtlicher Hinweis:
Amtsgericht Dortmund, HRB 20621
Vorstand: Wolfgang Neuhaus, Jens Wagener, Dr. Georg Pietrek
Aufsichtsrat: Prof. Dr. Burkhart Igel (Vors.) Stephan Grollmann,
Michael Neuhaus
