[news.eclipse.modeling.tmf] Keywords vs. tokens

[Marco <marco@xxxxxxxxxxx>]

Hi,

I do not really understand how the lexer works.

It looks like every string literal of my parsing rules are use to identify tokens with even higher priority than terminal rules. So e.g.:

From_import :
____'from' FQN 'import' ('*' | ((FQN ('as' FQN)?) (',' FQN ('as' FQN)?)*)) ;

destroys

terminal ARITHMETIC_OPERATOR : '+' | '-' | '*' | '/' | '%' | '&' | '|' ;

Is there an elegant way to avoid this?

I understand that in principle there is first lexing and then parsing. But is there a way where I can have different tokens depending of the context? I.e. Having words or symbols being a keyword in one section of a model and being normal text or just parts of another keyword in a different section?

I have for example a section where every line which does not include a certain symbol is treated as title. Currently I don't know better than listing all string literal and all terminal rules except the one symbol as alternatives for allowed characters. This hard to maintain.

Any hints or correction are very appreciated.

Best regards,
Marco