[cdt-dev] RE: [jdt-core-dev] Eclipse-conformant parser generator?

[Douglas Schaefer <dschaefe@xxxxxxxxxx>]

Hey guys, I'm cross posting this to
cdt-dev as well. Unfortunately we don't have a common place to discuss
issues common to language plug-ins (yet).

This is definitely one problem we're
faced with in the CDT. Even within C and C+ there are many language variants
supported by the various compilers out there (gcc included). How to deal
with that with our parser and AST is something we'll need to solve soon.
I like your idea of a cross solution - homo/hetero with an adapter. I have
also looked at ANTLR and am a big fan, but we founded we needed a lot more
imperative control over the parse to deal with the complexities of C++.
Having gone through it now, though, it might be worth taking another look.

There has also been a lot of discussion
on supporting multiple languages in Eclipse. As Ed Burnette mentioned,
there is a session at EclipseCon where we should hopefully discuss ways
to address this. Having a common framework for AST creation and navigation
that supported multiple languages would certainly help speed up some of
the development to support a new language, or even a new language variant.
It would also help plug-in developers build tools that can handle multiple
languages. Interesting ideas, we just need to find a way to coordinate
them and bring them to reality.

Cheers,
Doug Schaefer
CDT Dude

"Scott Stanchfield" <scott@xxxxxxxxxxxx>
Sent by: jdt-core-dev-admin@xxxxxxxxxxx

01/22/2004 11:03 AM

Please respond to
jdt-core-dev@xxxxxxxxxxx

To	<jdt-core-dev@xxxxxxxxxxx>
cc
Subject	RE: [jdt-core-dev] Eclipse-conformant parser generator?

I should also note that what I suggest here leverages
homogeneous AST nodes.
Heterogeneous nodes would be quite a bit more work... (in case someone's
not
familiar with the terms, homogeneous means all of the nodes are of the
same
type, and heterogeneous means there are specific types for specific nodes)

With homogeneous nodes and a tree walker grammar, you really don't need
heterogeneous, and the AST construction is greatly simplified.

Personally I prefer homegeneous nodes, as changes require only a change
in
the grammar & tree-walker grammar, and not changes to the node definitions.
It's much more dynamic.

The current JDT AST is heterogeneous. If the plan is to use heterogeneous
nodes for ASTs for other languages, this would make more generic automation
much more difficult.

[Of course a possible cross solution could be to use homogeneous nodes
for
the actual representation, with adapters that make each node look
heterogeneous. Once an adapter has been accessed, it returns heterogeneous
adapters for any children, so it all appears to be hetero.]

Not that I'm recommending we change the JDT ASTs; but if we're looking
for
an easier way to bring in other languages, I think homogeneous ASTs might
be
more suitable.

Thoughts?
-- Scott

> -----Original Message-----
> From: jdt-core-dev-admin@xxxxxxxxxxx 
> [mailto:jdt-core-dev-admin@xxxxxxxxxxx] On Behalf Of Scott Stanchfield
> Sent: Thursday, January 22, 2004 10:04 AM
> To: jdt-core-dev@xxxxxxxxxxx
> Subject: RE: [jdt-core-dev] Eclipse-conformant parser generator?
> 
> Interesting idea... Probably more of a general eclipse core 
> concept rather than just a jdt concept, though.
> 
> ANTLR has pluggable code generators (which can be subclassed 
> -- I've done that in the past when I integrated it into VAJ). 
> Overall this shouldn't be hard to do.
> 
> Of course, there's a learning curve associated with tools 
> like ANTLR, mainly getting in the mindset of declarative 
> programming via a grammar. I've got an old ANTLR tutorial at 
> http://javadude.com/articles that's a good start, but it 
> really comes down to mindset. But if you have to write a 
> parser anyway, much better to write one using a generator (IMHO).
> 
> To make it easier, I'd recommend that we look to ANTLR's 
> automatic AST generation capabilities. I haven't tried 
> overriding these in ANTLR, but I did it for PCCTS, ANTLR's 
> precursor, quite easily. Should be pretty easy with ANTLR as 
> well. If I recall, ANTLR has some factories for creating the nodes.
> 
> For example, a structure like the following (off the top of 
> my head) in a grammar
> 
>     FOR^ LPAREN! (initExpr)? SEMI!
>                  (testExpr)?
SEMI!
>                  (updateExpr)?
>          RPAREN! 
> 
> would automatically create an AST as follows:
> 1) throw out the LPAREN, RPAREN and SEMIs (the "!" says
> "don't include in
> AST")
> 2) promote the FOR to be the current parent node (the "^"
> says "promote me")
> 3) add the subtrees generated by initExpr, testExpr and 
> updateExpr to the current parent node
> 
> These concepts should be compatible with any AST 
> implementation; it's just a matter of determining which 
> objects to create for the nodes and how to connect them.
> 
> ANTLR.org is down right now, so I wasn't able to take a peek 
> at the generator code to see how easy this would be to do.
> 
> 
> Further, once the ASTs exist, it might also be interesting to 
> create a tree walker for the resulting ASTs. ANTLR has 
> tree-walker grammars, but they're based on ANTLR tree 
> structures. I'm pretty sure it could be adapted pretty easily 
> into reading other tree structures, such as ASTs in Ecilpse.
> 
> - Scott
> 
> (BTW: I used to work for Terence Parr, ANTLR's creator, and I 
> wrote the ParseView debugger for it. One of these days I need 
> to update and integrate the debugger with eclipse. Ter loves 
> eclipse, too...)
> 
> > -----Original Message-----
> > From: jdt-core-dev-admin@xxxxxxxxxxx
> > [mailto:jdt-core-dev-admin@xxxxxxxxxxx] On Behalf Of Eric Bodden
> > Sent: Wednesday, January 21, 2004 9:26 AM
> > To: jdt-core-dev@xxxxxxxxxxx
> > Subject: [jdt-core-dev] Eclipse-conformant parser generator?
> > 
> > Hello!
> > 
> > I was just thinking about how the actual build process for the
> > building the Java model works at the moment. Were all these classes
> > coded by hand or do/did people any parser generators in 
> order to build 
> > them?
> > 
> > I am just asking because I think it would be a great strategic
> > advantage vor Eclipse over other IDEs if somebody had a tool
could 
> > generate for any given input grammar a an appropriate parser
/ AST 
> > that is automatically Eclipse-conformant meaning it would 
> conform to 
> > conding standards, automatically subclass / implement necessary
> > superclasses / interfaces etc. I believe that would make 
> thinks like 
> > e.g. the current development of the C++ model or AspectJ model
much 
> > easier.
> > 
> > I think i general such a tool should not be too difficult 
> to develop 
> > if somebody used ANTLR or something like that.
> > 
> > What do people think about that?
> > 
> > Eric Bodden
> > --
> > www.bodden.de
> > 
> > 
> > _______________________________________________
> > jdt-core-dev mailing list
> > jdt-core-dev@xxxxxxxxxxx
> > http://dev.eclipse.org/mailman/listinfo/jdt-core-dev
> > 
> > 
> > 
> 
> 
> _______________________________________________
> jdt-core-dev mailing list
> jdt-core-dev@xxxxxxxxxxx
> http://dev.eclipse.org/mailman/listinfo/jdt-core-dev
> 
> 
> 


_______________________________________________
jdt-core-dev mailing list
jdt-core-dev@xxxxxxxxxxx
http://dev.eclipse.org/mailman/listinfo/jdt-core-dev