Re: [imp-dev] Integrating with existing Eclipse-based IDEs - what can IM

[Stan Sutton <suttons@xxxxxxxxxx>]

Hello Clemens,

Thanks for giving the more complete
description of your situation, that is helpful.

I don't believe that org.eclipse.imp.analysis
will directly give you the answers that you're looking for regarding type
determination/inferencing.  I can't speak as a developer/user of org.eclipse.imp.analysis,
but In general analysis would be based on language specific logic, and
IMP is largely premised on language independence.  IMP provides a
language-independent framework, base classes, and service skeletons for
the various IDE services.  The language/IDE developer must provide
language-specific details of representation and logic in order to complete
the service implementations.  So I believe that org.eclipse.imp.analysis
will provide generic data structures and concepts that would be useful
for various analysis problems, but you would need to implement the language-specific
logic to perform the analysis.  (I hope that someone who has experience
with org.eclipse.imp.analysis can give a more refined view as appropriate.)

As you stated, services such as reference
resolver, content proposer, documentation provider, and hover helper are
indeed helpful when writing code.  These are all found in the Eclipse
JDT, for example, and we have implemented them for several of our IMP IDEs.
 IMP does not generate full implementations of these services automatically--the
implementations must be completed by an IDE developer for a particular
language, making use of IMP-provided base classes and IMP-generated implementation
skeletons.  The implementations of different services may rely on
different amounts of analysis.  Content proposal may require fairly
elaborate analysis; hover help may be as simple as looking up some documentation
based on the type of the selected AST node.  How difficult a service
is to implement depends on the complexity of the language and the level
of functionality desired.  There are several examples of all of these
services in IMP-based IDEs that are included in the IMP release and code
base that are available on Eclipse.org.  There is one for the LPG
grammar specification language (org.eclipse.imp.lpg.runtime) and another
for the IMP preferences specification language (org.eclipse.imp.prefspecs),
among others.

Based on your description,
I gather that one thing you're considering is providing additional analysis
capabilities to plug in to existing extension points in EPIC or DLTK-based
IDEs.  Presumably you can implement those analyses in any way that
you want, so you could probably use org.eclipse.imp.analysis, if you find
that it is helpful (bearing in mind that you still have to program the
analysis logic).  Unfortunately, I can't give you advice on how to
use it, though.

> I see. So if for example I found that the AST
support for DLTK's 
> _javascript_ IDE implementation lacking I could use LPG to create a
parser 
> to get an AST which I can use with org.eclipse.imp.analysis to make
use 
> of the features provided by that?

Yes, that is exactly right.
 If you have the grammar for a language, and you can represent that
grammar in the LPG grammar-specification language, then you can use LPG
to generate a parser and AST representation.  Putting an existing
grammar into the particular representation used by LPG is probably not
conceptually difficult--perhaps you just have to transcribe the grammar
rules into a different syntax.  But you also need to assure that LPG
will generate a parser that is adequate to handle any semantic complexities
of the grammar.  LPG generates LALR(k) parsers.  If a particular
grammar can be handled by such a parser then using LPG for it should be
fine.  If there are additional complexities in the grammar that take
it out of these categories, then some additional customization of the parser
may be necessary.  (For more information about the use of LPG to generate
parsers for various languages, contact Philippe Charles, pgcharles@xxxxxxxxxxx.)

As Jin points out in a separate message,
if you have an existing parser and AST representation, and you can use
these to implement a small number of fairly simple interfaces, then you
can get the rest of IMP without having to generate a new parser or AST
representation.  The interfaces that need to be implemented are mainly
(if not entirely) those
defined in the org.eclipse.imp.parser package of the org.eclipse.imp.runtime
project.

If you want to support the
editing of scripts separately from EPIC or DLTK, then parts of IMP in addition
to org.eclipse.imp.analysis may be helpful to you.  That is, you could
create your own language-customized IDEs for script
editing that would have syntax coloring, outline view, text folding, and
other services (such as those discussed above and listed in the earlier
message).  You can do this fairly straightforwardly if you have the
parser and AST representation and have implemented the appropriate interfaces
for those (or you have used LPG to generate the parser and AST representations).
 Otherwise, the most complicated part is implementing some of the
more complicated analyses, as we've been discussing.  But if you have
to implement the analysis yourself (even for use with EPIC or DLTK) then
you could presumably also use those analysis implementations with IMP.
 IMP does not provide any support for executing scripts, but I gather
that your company will provide that.  If you have the appropriate
parser and AST representation, and also the analysis algorithms, then IMP
will give you the leverage to create a nice customized language-specific
editor with relatively little additional work.

There are some real trade-offs between
the approaches that you are considering.  I don't know enough to make
a recommendation on specific alternatives, but I hope you find this helpful
in making a decision.

Regards,

Stan

Stan Sutton, Ph. D.
IBM T. J. Watson Research Center
19 Skyline Drive, Hawthorne, NY 10532 USA
telephone:  1-914-784-7316, FAX:  1-914-784-7455, T/L 863
e-mail:  suttons@xxxxxxxxxx, Stan Sutton/Watson/IBM@IBMUS

Clemens Anhuth <clemens@xxxxxxxxxxxxx>
Sent by: imp-dev-bounces@xxxxxxxxxxx

04/15/2009 11:13 AM

Please respond to
IMP Developers List <imp-dev@xxxxxxxxxxx>

To	IMP Developers List <imp-dev@xxxxxxxxxxx>
cc
Subject	Re: [imp-dev] Integrating with existing Eclipse-based IDEs - what        can IMP do for me?

Stan Sutton wrote:

[...]
> I don't have a very clear picture of what you're trying to do.  As
Jin 
> suggests, IMP, DLTK, and EPIC are different kinds of tools or 
> platforms and have different positions with regard to language support
> (representation, tooling, runtime/debug).
>
> Can you say more about what your third API is and what the integration
> is supposed to achieve?  

Hello Jin, Stan,

sorry, that should have been "third party API".

My company provides a software that comes with support for running 
scripts in Perl, Python, Tcl and _javascript_. The current IDE for writing
these scripts will be replaced with an Eclipse-based IDE. The first 
choices for this so far are DLTK and EPIC. (The plugins that make up the
new IDE will be usable separately, so that one can install the new IDE's
functionality into an existing installation of EPIC or DLTK.)

The software exposes the same API to any of the script language 
interpreters. The API does not exist in form of script files. (As such
the situation is similar to that of writing _javascript_ scripts for Adobe
or Microsoft products, each having their own APIs and extensions 
exposed/embedded to/in their interpreters, ready to be used in scripts
without further configuration.)

Writing the scripts can be done with any text editor or IDE, but then 
code completion support for that particular API is missing.

We want to offer that code completion support for our API in DLTK and 
EPIC, but also in any other Eclipse-based IDE, if possible. (Where 
possible we want to integrate our own code completion seamlessly, like
can be done with the respective extension point in DLTK. Where seamless
integration is not possible we would like to offer our own code 
completion functionality to be triggered with a key combination.)

To me it seemed that this is maybe where org.eclipse.imp.analysis has 
something to offer.



>
> I haven't worked on org.eclipse.imp.analysis or made use of it.  It
> should help you in implementing AST-based analyses.  I can't
comment 
> on its readiness for use, though.

OK.

>
> More broadly, IMP is not an IDE but is metatooling mainly to 
> facilitate the development of IDE services or features.  Many
of these 
> are oriented around the editor but some are not.  The IDE developer
> must provide basic parsing services and an AST representation.  The
> IMP runtime for editing and other services generally rely on these.
>  Strictly speaking, the language tooling and AST representation
are 
> not part of or provided by IMP, although IMP does come "bundled"
with 
> the LPG parser generator which can be used to generate a parser and
> AST representation.  Other approaches to parsing/AST support
can also 
> be accommodated.

I see. So if for example I found that the AST support for DLTK's 
_javascript_ IDE implementation lacking I could use LPG to create a parser
to get an AST which I can use with org.eclipse.imp.analysis to make use
of the features provided by that?



> Given an implementation of the parsing services and AST 
> representation, IMP provides leverage for development of IDE services.
>  The image below gives an indication of services currently addressed
> (this is the IMP contribution to the Eclipse "New" menu).
 It's not 
> clear if any of these would be helpful for your purposes.  However,
> even if they don't address your main focus you may still find some
of 
> them nice to have.

Reference resolver, Content proposer, Documentation provider, Hover 
helper all sound like they could be related to code completion or to 
providing helpful information while writing code, more or less directly,
but that's only from guessing by their name.



I guess the main problem I am still looking a good solution for is type
detection (is that called type inference?) to be able to provide code 
completion of members of "obj" in cases like this one:

  var objName = "name of an object";
  var obj = findObject(objName);

What seems tricky to me about this is that the "type" returned
by 
findObject() (which is not implemented as/in a script file) is 
determined by the actual name passed to it.

This example has one indirection to passing the name to findObject() 
(via the variable objName) and I think I can imagine how to analyse the
AST to get the actual name passed. But I guess I wonder if that is still
possible when there are more indirections, and whether IMP or 
org.eclipse.imp.analysis already has some feature to do or help doing this.



With best regards

Clemens Anhuth

_______________________________________________
imp-dev mailing list
imp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/imp-dev