Re: [eclipse.org-architecture-council] Javascript: a bug that makes me r

[Michael Scharf <eclipse@xxxxxxxxx>]

On 2015-07-07 8:11, Sven Efftinge wrote:>
>> In addition to a common AST, we should also think about a *very* simple
>> API that allows external tools to contribute to any editor highlights
>> based on the position in the file (line:column data).

> *Very* simple means you need to leave out more involving but useful
> features.  E.g. content assist is not simply proposals for some text
> and offset. There are interesting advanced features Jface provides
> like "linked editing”. I mean there must be a reason we have
> ICompletionProposalExtension6 :-)

For linked editing you need to know the scope of identifiers.

> We are currently reimplementing all the IDE services for Xtext such
> that they can be used with other ‘smart’ editors, and it seems like
> simple is not always good.  Btw, in that sense Xtext also becomes an
> external tool to other editors, similar to what tern is for JS.

+1


>> Most "smart editors"
>> have have APIs to run external tools, once a file is saved and they parse
>> the output of stdout. The output is simple position information with
>> some data to be visualized. With such an interface you can
>>
>> - run lint tools to find all kinds of errors
>> - run build tools to visualize build errors and warnings
>> - run code coverage
>> - run profile tools
>> - run unit test
>> - run tool to get syntax and semantic highlights
>> - run something like http://ctags.sourceforge.net for code navigation
>> - run an external tool for code completion or use a generic
>>   code completion tool based on codetrails
>> - run tools, that automatically format code
>> - follow standards like editorconfig.org

> If all these tools only share that they expect the same textual
> input, than there is a lot of duplicated work (parsing, linking,
> etc.) going on for each of them.

Most of the tools mentioned above run are of different nature and
nothing prevents you form having a single tool that does syntax
highlight and code completion. Profilers, code coverage, build, test
tools and style checkers are running independently, yet most of the
tools just provide information about locations in files. If there
would be a simple mechanism to add external tools that parses the
output and show them in any editor would be very helpful.

My favorite example is makefile support: it is integral part of CDT.
Heck! Why can't I use make for my pure JavaScript project without
installing CDT?


> That might work if that work is super cheap, but that is usually not
> the case.

Running external parsers (actually having a parser service running,
that got notified when files are changing) worked 25 years ago quite
well, when workstations had single core processors running at 20Mhz.
Today each of the 8 or 16 cores is 100 times faster, why should this
suddenly be a problem ;-)....


> Also you need to think about something like an index (could be
> external, though), so you can provide proper proposals, navigation
> and affection detection.

If we talk about strongly typed languages with a well defined module
system, parsing and linking is required to get a complete picture
of the system. It turns out that there are many languages and systems
that are not that well defined. Tools can not easily figure out all
dependencies and linking between the code. A lot of 'linkage' is done
in the build process, by some scripts, by some conventions deep inside
a library. It turns out that a good text search tool with filtering on
working sets (or scopes if you use JetBains IDEs) is often more
reliable than tools that do not understand the (often implicit)
linkage between files and languages.

The real problem starts with the fact that each *DT maintains its own
index and there is no higher level structure that allows defining
dependencies between sub-parts of the system. In some way there has to
be a hierarchy of scopes that goes beyond a single language.

If there would be a way to define an 'architecture' of the system,
where users can provide hints on which files are linked together then
text based code completion and navigation can get pretty far, I
believe.

For example, WebStorm is pretty good at doing code completion in
javasctipt and coffescript, but it often gets totally confused by some
unrelated files somewhere in the project. I urgently want to tell the
IDE which files are actually relevant for the file I am currently in.

The assumption that we can write tools that can automatically figure
out all kinds of linkages in multi-language projects with complicated
build processes and dynamically typed languages is an illusion.

A more pragmatic approach would be helpful here which should not
prevent anyone form writing 'perfect' tools for well defined systems.


> That said, I think plain Eclipse does offer hooks to do the things
> above.  Do you mean simple in the sense, that you don’t want to
> write Java and build a plug-in?

Yes!

> Why?

Many developers of tools language X that should be integrated into
eclipse really know that language. If there is an API (plus a few
configuration files) where they just have to provide some well defined
text output, it is easier for them to write such a tool in language X
(based on something existing) than to learn Java and how to write
eclipse plug-ins, create an update site, publish the result to
the eclipse market place.

It took me a 2 hours to write and publish my first node module
(including learning how to publish npm modules). It actually is a hack
for JetBrains IDEs
  https://www.npmjs.com/package/mocha-for-jetbrains
Here I decided it was *much* easier to 'decorate' the external tool
instead of trying to write of modify the Jetbrains extension.

Setting up an eclipse project take significantly longer, especially
if you have not done it before.

> This needs to be done only once for every language so many users can
> simply install the solution. Or should every develop configure the
> stuff above on her own?

It needs to be done once. But a tool written in X can be used in the
build system, in editors and other IDEs. Any time spend writing an
eclipse plugin is 'wasted time'. So, if I would integrate language X I
would do as much as I can in X. Most of that work can be used in any
other environment. Instead of writing my tooling in "eclipse-Java"
which would not even run in anther Java based IDE (like JetBrains).

If I have some of my own 'analysis' scripts, I want to integrate them
without writing a plug-in. Just a setup file in my workspace
should be sufficient.

> Regarding JS and tern, as I think you are mostly concerned about that:
> I think an editor that can be configured with syntax coloring, is
> very different to a plug-in that fully leverages the power of tern.

You might be right!

> For instance sublime seem to do only proposals with tern, while tern
> itself offers many more interesting features. It is not a simple
> external tool, it is a full provider of JS-specific tool
> support.

That's fine. But writing all the infrastructure from scratch is
annoying.

> IMHO it is a good idea having a dedicated plug-in that is
> tailored to leverage exactly that (like Max mentioned) rather than
> building a generic framework for possibly existing external tools.

Both approaches have their use cases....

In some way each *DT is a silo in eclipse. There is almost no code
sharing between them. There is not even a common index or a common way
to integrate external tools that provide information about file
locations that works with all *DTs. Xtext simplifies that process by
generating a good portion of that silo for each new language. But
it's still silos....

Michael

> Cheers,
> Sven
>