[ide-dev] Code editors in the Eclipse (IDE)

[Tom Schindl <tom.schindl@xxxxxxxxxxxxxxx>]

Hi,

*Disclaimer: This post is going to be long I hope you follow until the end*

There seems to be a lot of momentum to make simple code-editors in
Eclipse possible.

As I've been working since some time on this topic I'd like to share
with you my take on this (Max already mentioned my name in some of the
threads).

*Let's start with a requirements*
=================================

Simple requirements
-------------------

The first and most important things needed are:
- syntax highlight (lexical!)
- block detection (for curly brace language this means matching
  {},[],() pairs) used for highlighting and code-folding (although I
  don't really see code-folding as an important feature at all)

The important thing for those 2 features is:
a) they must be fast
b) they must be fast

This means there's no need to build an AST hence no language-server,
compiler, ...

There are solutions available for that already:
- Angelo has TextMate support
- LiClipse has TextMate support & its custom spec format
- e(fx)clipse has its own format (who misses code folding as of today)

Although I'm not against having TextMate I repeat that the most
important thing for lexical highlighting is speed hence I'm not sure the
TextMate route is the best way.

Having it as a fall back strategy if not better format is available is
ok but for maximum performance we need something who aligns 100% to the
way Eclipse-Text is working.

Complex requirements
--------------------

Once we have something that matches the simple requirements we can work
integrating the more complex ones who are (once more in order of
importance):
- Auto-Complete
- Error-Reporting
- Documentation Hover
- Context-Information
- Code navigation
- Semantic-Highlighting (there might be languages like DSLs where
  semantic highlighting is even more import than anything else eg CSS)

Most of those can be made possible using Language-Servers and their
protocols [1,2] (although I would not directly built in the support for
a specific protocol into Eclipse).

Even more Complex requirements
------------------------------

- Connection to Debuggers, ... - no idea about that yet


*How to implement*
==================

Bug 496114 suggest to implement this support by making TextEditor
smarter and Bug 496119 suggest to implement a PoC based on JSDT.

Doing a PoC using JSDT is ok, although I'm uncertain what one would
learn from that beside that: Yes you can implement ContentAssist based
on it and yes you can do Error-Markers but that should not be a suprise
given that there are PoC demostrating this even at the lower-level APIs
of SourceViewer.

On to bug 496114. I would stay away of doing the implementation on
TextEditor, it carries so much history with it and what makes it for me
even less attractive is that it is so tightly bound to the Eclipse IDE
and the compat layer so that using it for custom IDEs (who are getting
more on more built on pure e4) a none-starter.

Finally implementing all this directly into TextEditor will most likely
not work because one will interfere with subclasses like JDT, Xtext, ...
and IIRC the editor uses SourceViewer as now (subclasses like JDT
overwrite that) and eg for Folding you need to have a ProjectionViewer.

Leave it alone and start with a clean code base who takes the lessons
learned since then and implement a new one:
* who uses services
* who uses dependency injection
* who can run in
  - the Eclipse IDE
  - an e4 application with core.resources
  - an e4 application without core.resources
  - an plain java-application without core.resources
  - an e4 application with SWT
  - an e4 application with *fill-in-your-favorite*
* don't use preferences to carry colors, they should come from a theme

Now to finish up: Can this be done, is there a PoC.

Yes there is:

SWT:
https://github.com/BestSolution-at/code-swt

JavaFX
https://github.com/BestSolution-at/dartedit

It uses the Dart-Language-Server and not the M$-Language-ServerAPI
because this work is older than a year but the concept is so so similar
that you can be sure it will work with it.

Beside the few SWT bits in code-swt project the APIs are all coming from
the e(fx)clipse project (who is an Eclipse.org project) and unlike its
name we have designed and bundled things in a way that they can run
without JavaFX and Eclipse.

We have services for almost all of the above features and split them
*always* in 2 parts:
- core-service who is reponsible to provide the data
- ui-service / UI-Technology who is responsible to present the data

What type of services do we have so far:

* Lexical highligthing
- Document Partitioning == IPartitionTokenScanner
- Damaging / Repairing == IPresentationReconciler

=> we have a concrete implementation for them based upon our
   declarative highlighting language but if there's a service with a
   higher-ranking this would win the game available for JavaFX and SWT

* Auto-Complete
- Core-Service OSS implementation for Dart & Typescript
- (JavaFX only) UI-Service OSS implementation for Dart & Typescript

* Error-Marker
- Core-Service OSS implementation for Dart & Typescript
- (JavaFX only) UI-Service OSS implementation for Dart

* Code-Navigation
- Core-Service OSS implementation for Dart & Typescript
- UI implememtation for JavaFX-Editor

* Hover
- Core-Service OSS implementation for Dart & Typescript
- (JavaFX only) UI-Service OSS implementation for Dart & Typescript

* Pair-Matching
- Core-Service OSS implementation for Curly-Brace languages
- (JavaFX only) UI-Service OSS implementation for Dart & Typescript

Finally what languages have we already look at and we know we can
implement with our APIs and their headless servers:
- Dart
- Typescript
- Java (we built a small headless-server with JDT-Core)
- JavaScript (we leverage LanguageService API)
- Rust
- Go

Tom

[1]https://github.com/Microsoft/vscode-languageserver-protocol
[2]http://htmlpreview.github.io/?https://github.com/dart-lang/sdk/blob/master/pkg/analysis_server/doc/api.html

-- 
Thomas Schindl, CTO
BestSolution.at EDV Systemhaus GmbH
Eduard-Bodem-Gasse 5-7, A-6020 Innsbruck
http://www.bestsolution.at/
Reg. Nr. FN 222302s am Firmenbuchgericht Innsbruck