[news.eclipse.technology.dltk] Re: ASTNode sourceStart and sourceEnd - r

[Andrei Sobolev <haiodo@xxxxxxxxx>]

Hi Chuck,

In the beginning of DLTK and Python support, ASTNode class was constructor with ANTLR Token parameters.
Then we move to Eclipse, we removed ANTLR dependency from core and created this wrapper class. It is only used from
Python parser. Better idea it is not to use it at all, as we do in Tcl and Ruby.

I suppose for some problems you described we have for some Python statements.

I suppose better idea for you is not to use DLTKToken at all. I've marked this class as deprecated in HEAD.

Best regards,
Andrei Sobolev.

> Every DLTKToken has the members:
> line
> column
> text (from which you can get text.length())
> 
> Please note that line is *never* read.
> 
> 
> ASTNode has the members:
> sourceStart
> sourceEnd
> 
> These are initialized in the constructor to be:
> 
> protected ASTNode(DLTKToken token) {
>   this.sourceStart = token.getColumn();
>   String tokenValue = token.getText();
>   if (tokenValue != null) {
>     this.sourceEnd = this.sourceStart + tokenValue.length();
>   }
> }
> 
> If this node is all on the first line, this makes sense.
> If the node does not start on the first line or spans multiple lines, this 
> doesn't make sense.

> Somehow you need to determine a running offset from the start of the source 
> module.
> I don't see where/how that is done.
> 
> Later, in ASTNode, is getSourceRange:
> 
> protected ISourceRange getSourceRange() {
>    return new SourceRange(this.sourceStart(), this.sourceEnd() - 
> this.sourceStart() + 1);
> }
> 
> Here's what the documentation for ISourceRange says:
> A source range defines an element's source coordinates relative to its 
> source buffer.
> 
> int getOffset()
>           Returns the 0-based index of the first character of the source 
> code for this element, relative to the source buffer in which this element 
> is contained.
> 
> int getLength()
>           Returns the number of characters of the source code for this 
> element, relative to the source buffer in which this element is contained.
> 
>