Re: [cdt-dev] lpg question

[David Sariel <datosar@xxxxxxxxx>]

Hi Mike,

Thanks a lot for the answer!

 

"Reduceing” my new language constructs into ones that already exist in C seems to me the fastest way to make things working. However, I’m still novice and some advice will be appreciated in order to make sure I’m in the right direction:

 

While trying to “reduce” the language constructs I played a little bit and changed functions to be declared “func foo(in param1, out param2, inout param3,…). In order to make that I used the rule

simple_type_specifier_token

::= interface_type -- Dropping all others (int, char, etc)

 

interface_type

    ::= 'in'       

      | 'out'

      | 'inout'

 

 

But code completion assumes that all params are of type int and suggests the suggestion “foo(int param1, int param2, int param3): int”. And in outline window I see foo(,,) - with no parameter types at all. The question is where the outline window and code completion decide which type the parameters are.

 

----------------------------------------------------------------------------------------------------

The second question is that in order to add func declarator I’ve dropped the rule     

    direct_declarator

    ::= basic_direct_declarator

 

And instead  added the rules and terminals:

    direct_declarator

    ::= func_directive basic_direct_declarator

     

    func_directive

      ::= 'func'

     

   FuncDirective ::= 'func' – new terminal

 

 

Also, func(TK_FuncDirective) was added to the UPCKeyword enum.

 

 

 

But, while parsing the file that has the only line

i	n	t		f	u	n	k		f	o	o	(	)	{	}
1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16

 

I arrive to this in the first rule reduce:

external_declaration ::= ERROR_TOKEN (Offset: 0, Length: 8 calls consumeDeclarationProblem)

 

And the following AST was build:

 

Base Extensible Language AST:

org.eclipse.cdt.internal.core.dom.parser.c.CASTTranslationUnit (0,16)

  org.eclipse.cdt.internal.core.dom.parser.c.CASTProblemDeclaration (0,8)

    org.eclipse.cdt.internal.core.dom.parser.c.CASTProblem (0,8)

  org.eclipse.cdt.internal.core.dom.parser.c.CASTFunctionDefinition (9,7)

    org.eclipse.cdt.internal.core.dom.parser.upc.ast.UPCASTSimpleDeclSpecifier (0,0)

    org.eclipse.cdt.internal.core.dom.parser.c.CASTFunctionDeclarator (9,5)

      org.eclipse.cdt.internal.core.dom.parser.c.CASTName (9,3)  foo

    org.eclipse.cdt.internal.core.dom.parser.c.CASTCompoundStatement (14,2)

 

Why “int func” is considered a problem? Am I missing something?

  

----------------------------------------------------------------------------------------------------

 

And the last question is just in order to assess times: how much time it took to design/write/test UPC?

Thanks a lot for the help

David

 

On Wed, Sep 30, 2009 at 5:00 PM, Mike Kucera <mkucera@xxxxxxxxxx> wrote:

Hi David,

If I understand what you're asking, the problem is that the CDT AST does not support statements as children to the translation unit node, but your language does.
I think there are a couple options you could explore. You could create some kind of "statement declaration" node that implements IASTDeclaration and add it to the translation unit. Or you could provide your own translation unit node that allows statements. Or you could wrap the statements in a dummy function declaration node.

In any case the consumeTranslationUnit() method can be overridden. Just create your own subclass and override that method and any others you need to, then in the grammar file set the $build_action_class macro to the name of your class. I think in an extending parser its ok to make direct reference to any new AST node types that you've created. The main purpose of the node factories is to allow the CDT parsers to share parsing rules between C and C++ while generating the correct AST. You don't have to follow this pattern in your own parser if it gets in your way.

But there is a deeper problem here. At the end of the day CDT only understands its own built-in AST nodes. If you generate an AST that differs significantly from what CDT expects then a lot of the features won't work. I got around this in the UPC parser by having all my new AST nodes implement node interfaces already predefined in CDT. For example UPC has a new kind of for loop that has an extra _expression_ in the header. I created a new kind of AST node for that but it extends CASTForStatement, so CDT usually treats it as a normal for loop. You may find that you have to "reduce" or "desugar" your new language constructs into ones that already exist in C otherwise CDT will get confused. If you really get stuck there is the possibility of making changes directly to CDT to support your language. We've done this in the past with a language called POP C++ or something like that.



Mike Kucera
Software Developer
IBM Eclipse CDT/PTP Team
mkucera@xxxxxxxxxx

David Sariel ---09/29/2009 01:05:20 PM---Hi Mike,

From:	David Sariel <datosar@xxxxxxxxx>
To:	"CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Date:	09/29/2009 01:05 PM
Subject:	Re: [cdt-dev] lpg question

---

Hi Mike,
Thanks a lot for the answer!
I have a written a grammar that was received by adding/dropping rules
in UPC/C99 and the generated parser seems reducing rules correctly.

Now I want to reuse as much as possible the parser action classes
(C99BuildASTParserAction, BuildASTParserAction, AbstractParserAction)
in the manner that will cause the
1) Syntax highlighting
2) Code outline
3) Code completion
to work with the new language (I have the part of keyword highlighting
already working).

The problem is that the language I’m working with has no declaration
and type sections that are presented in C99. Bellow are the rules that
bypass the declaration and type rules as I’ve defined:

119  172   directive ::= align_directive               (NEW)
120  173   directive ::= ds_directive                  (NEW)
121  174   directive ::= dcb_directive                (NEW)
122  175   directive ::= dcw_directive                (NEW)
123  176   directive ::= dcl_directive                 (NEW)
124  177   directive ::= list_directive                 (NEW)
125  178   directive ::= nolist_directive              (NEW)
126  179   directive ::= global_directive             (NEW)
127  180   directive ::= const_directive              (NEW)
128  181   directive ::= register_directive           (NEW)
129  182   directive ::= reserve_directive            (NEW)
130  183   directive ::= strict_directive               (NEW)
131  184   directive ::= nonstrict_directive         (NEW)
132  185   directive ::= sync_directive               (NEW)
133  186   directive ::= nosync_directive            (NEW)
134  187   directive ::= functioncall_attr             (NEW)
135  188   directive ::= func_declarator              (NEW)
136  189   directive ::= outofline_directive           (NEW)
137  190   directive ::= struct_or_union_specifier (NEW)
138  191   directive ::= enum_specifier               (NEW)

139  93    expression_statement ::= ;
140  94    expression_statement ::= expression_in_statement ;
141  154   expression_statement ::= directive ;   (NEW)

158  213   section_definition ::= SectionDirective identifier_token
compound_statement  (NEW)

159  79    statement ::= labeled_statement
160  80    statement ::= compound_statement
161  81    statement ::= expression_statement
162  82    statement ::= selection_statement
163  83    statement ::= iteration_statement
164  84    statement ::= jump_statement
165  149   statement ::= include_directive              (NEW)
166  150   statement ::= inline_func_definition        (NEW)
167  151   statement ::= func_definition                 (NEW)
168  152   statement ::= section_definition             (NEW)
169  153   statement ::= ERROR_TOKEN

170  92    block_item ::= statement                       (NEW)
171  90    block_item_list ::= block_item
172  91    block_item_list ::= block_item_list block_item

173  211   translation_unit ::= block_item_list
174  212   translation_unit ::=
175  0     $accept ::= translation_unit
176  2     <empty> ::=

 

Hence, one of the problems I have is how to override the
BuildASTParserAction#consumeTranslationUnit method  to add/get
statements to the TranslationUnit instead of
addDeclarations/getDeclaration methods (the problem is that
INodeFactory is annotated by @noextend, @noimplement).

How to overcome this?
Thank you,
David

On Mon, Sep 14, 2009 at 5:34 PM, Mike Kucera <mkucera@xxxxxxxxxx> wrote:

Hi David,

I take it what you're trying to do is have LPG print out which reductions are actually happening during a parse so that you can debug your parser.

As far as I know there is no way to automatically get LPG to do that for you, so you need to do it yourself. You'll notice that the information you need is at the top of the *.l files. So to make it work I wrote a little script that reads the *.l files and generates a Java class that contains a giant String array where the index is the rule number and the element is the rule. Then I added a couple lines of code to the top of the ruleAction() method in the generated parser that uses the array to print out the rules that are being fired.


Mike Kucera
Software Developer
IBM Eclipse CDT Team
mkucera@xxxxxxxxxx

David Sariel ---09/14/2009 04:12:35 AM---Hello,


From:	David Sariel <datosar@xxxxxxxxx>
To:	"CDT General developers list." <cdt-dev@xxxxxxxxxxx>
Date:	09/14/2009 04:12 AM
Subject:	[cdt-dev] lpg question

---




Hello,
yacc (bison) supports --verbose and --debug options allowing automatic
generation of debug outputs like:

Reducing via rule 93 (line 329), statement -> statement_list
...
Reducing via rule 185 (line 601), statement_list -> translation unit
...

Does lpg analogs (-verbose and -debug switches) allow the same? I'm
generating parsers with those options.
*.l files indicate that VERBOSE, DEBUG are options in effect. But
parser files generated are the same
as without -verbose and -debug, i.e. no debugging outputs were generated.

Am I missing something?

Thanks,
David
_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev



_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev

_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cdt-dev