[imp-dev] Re: [imp-commit] r21620 - in org.eclipse.imp.pdb/trunk: META-I

[Jurgen Vinju <jurgen@xxxxxxxxx>]

Hi Bob,

So far, Rascal definitely only supports "concrete syntax patterns" if
they are defined by SDF.
SDF is nice and all, and if you can find the time to re-implement your
front-end.... well, I don' think
that is feasible in a lot of cases. (In short, I agree with IMP's
earlier design decisions).

There is the possibility of connecting a fixed but open set of
parser generators and their respective AST formats, including SDF,
LPG, ANTLR, JavaCC. This is doable,
and it covers a significant amount of (domain specific) languages, but
not the real programming languages.
I.o.w., his still doesn't really solve the problem of allowing arbitrary AST's.

Rascal has really powerful matching and visiting constructs of
abstract syntax trees.
I would not like to give up just because we want to be technology
agnostic. I would like IMP to be technology
parametric. So I'd still like to talk about how we can connect these
constructs to arbitrary
different AST's.  I don't see how this could be done without
wrapping/bridging/converting the
existing AST's in one way or another. So, I'd like to try and find a
solution that "doesn't hurt so much".

I do suspect that there is no easy solution for this problem. The
problem is the width of these existing
AST hierarchies. They can be big. If there is no abstract interface to
all of the different classes
in a specific AST hierarchy to allow visiting and matching, then there
exists no short solution.

Let's keep our mind open: we don't necessarily need a wrapper API, we
could use source-to-source transformation, or aspect weaving,
or reflection, Java annotations, or Eclipse extension points, or
whatever, to get the job of matching the PDB's INode and IList
interfaces
to these existing AST API's. (The basic pattern matching in Rascal is
based on these interfaces).

The TOM tool takes an interesting approach, it is a pre-compiler for
pattern matching. It allows to define code snippets
that implement the basic operations needed for pattern matching on
each kind of AST: what is its name, what are its children?
Then, the TOM compiler generates pattern matching code that weaves in
these snippets at the right locations. The result
is that there is no wrapping API to maintain, which would also take a
run-time overhead. TOM comes with a number of tools
that generate the code snippets for particular front-end technologies,
such as SDF.

I also like the idea of Java annotations. Combined with some
source-to-source or byte-code transformation they could be used
to automatically add methods to existing AST classes to implement the
required interfaces. The job left to do is to annotate
each AST class with the information needed: which fields represent the
identity of the class, where can we find the children?

I'm not sure these are good approaches to take, but I just wanted to
give alternatives to a big wrapping API.

Question: does "technology agnostic" also mean that it should not have
to be Java?

Cheers,

Jurgen


On Tue, May 12, 2009 at 7:10 PM, Robert M. Fuhrer
<rfuhrer@xxxxxxxxxxxxxx> wrote:
> Hi Jurgen,
>
> The main (only?) problem with Rascal, as nice as it is, is that it only
> works with SDF-generated parsers. (Right?)
>
> We've had this Catch-22 problem regarding transformation for a long time
> now:
>
>  1) We'd like to support arbitrary parsers, but
>  2) We don't like the ugliness that results from wrapping parser-generated
> ASTs APIs to adapt them to a generic pattern matcher/transformer.
> [org.eclipse.imp.xform took this approach, and it's... well... somewhat
> repulsive. But maybe I'm being a bit squeamish here.]
>
> Perhaps we just have to bite the bullet and say that if you want a good
> matching/transformation story, you have to use SDF+Rascal.
>
> But if we do, we're crossing a threshold that's antithetical to IMP's stated
> goal of technology-agnosticism.
>
> I guess I'm not sure I'm ready to cross that threshold.
>
> On the other hand, perhaps we can do both: have an ugly but workable story
> for arbitrary parsers/ASTs, and a much nicer story for one or more specific,
> well-behaved forms of ASTs (like SDF).
>
> What do others think?
>
> IMO, this is a fairly important decision for the project.
>
> On May 12, 2009, at 11:10 AM, Jurgen Vinju wrote:
>
>> Hi Bob,
>>
>> I also think we need a summit. When would be good for you? Rascal is
>> good for transformation.
>>
>> Cheers,
>>
>> Jurgen
>>
>> On Tue, May 12, 2009 at 4:43 PM, Robert M. Fuhrer
>> <rfuhrer@xxxxxxxxxxxxxx> wrote:
>>>
>>> Hi Jurgen,
>>>
>>> BTW, we *still* need to get a decent story in place for transformation
>>> for
>>> IMP. I think we may need a "summit" of some sort to get the
>>> ball rolling on that front.
>
> --
> Cheers,
>  - Bob
> -------------------------------------------------
> Robert M. Fuhrer
> Research Staff Member
> Programming Technologies Dept.
> IBM T.J. Watson Research Center
>
> IMP Project Lead (http://www.eclipse.org/imp)
> X10: Productivity for High-Performance Parallel Programming
> (http://x10.sf.net)
>
>