[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [ide-dev] Code editors in the Eclipse (IDE)

Thanks, Bruno. Good points. Just one remark to avoid confusion:
Every language will need lexical syntax coloring in the way TextMate grammars provide them, because semantic coloring is too slow to be used alone.
So the classic strategy is to highlight lexically as you type and update styles when the semantic analysis has finished in the background (the reconciler in Eclipse-speech).

2016-06-20 14:37 GMT+02:00 Bruno Medeiros <bruno.do.medeiros@xxxxxxxxx>:

Seems like there are a lot of points, ideas, goals and desiderata being floated around and discussed here. It can be a bit hard to follow at times, at least for me. There is some things I'd like to point out though:

First, on the the issues of TextMate grammars. I'm not that familiar with them, but from what I hear in this discussion, TextMate grammar support might be useful for "simple", or "markup" languages like XML, JSON, _javascript_, etc. , but for big languages like Java, C/C++, Rust, Go, D, Swift, etc, it seems like support using TextMate might not be a waste. Let me explain why.

First of all there, in a basic way one can categorize language support into 3 layers of functionality:

1) lexical -> syntax highlighting, maybe editor auto-edits and bracket highlighting. You need some sort of lexer.
2) structural -> parse errors, outline and code folding. You basically need some sort of parser for this functionality.
3) semantic -> compile errors, find definition, code complete, find references, extract method, etc.. You need a semantic engine for this, even if not one as complete or accurate as an actual compiler.

From what I understand from this discussion, Textmate grammars help you with layers 1 and 2, but for 3 you need something else. Now, the thing is, virtually anyone who wants to work with a "big" language, they will want a full IDE, with support for layer 3 as well. People are not embedding Java, C++, Rust etc, in other projects like they would with XML, JSON, _javascript_ or even a scripting language like python.

So there needs to be at least some layer 3 support (semantic analysis), even if not as good as JDT. But, for any minimal layer 3 (semantic) support, you need a parser anyways, so you can trivially provide layer 2 functionality if you have any layer 3 support. (I'll give an real example with Rust)

So in effect TextMateonly helps us with layer 1 functionality... however layer 1 functionality is very simple to write (the language specific bits that is). I wrote the syntax highlighting lexing rules for the Eclipse IDEs for Go, Rust, and D, and for each IDE that was like 0.01% of the total effort involved for the whole IDE project. Basically half a day worth of work. Note: I'm talking only about the language-specific code here. Yes, the UI infrastructure (preference pages for each highlighting item, etc.) was much more complex and took more time, but that is framework code, not language-specific code.

As for the Rust example, in RustDT (https://github.com/RustDT/RustDT) the layer 2 functionality is provided by Rainicorn (https://github.com/RustDT/Rainicorn) a tool that displays parse errors, and the structure of the source file[1]. Rainicorn is a very simple tool that just re-uses the pre-existing official Rust parser, the only thing it does is basically serialize the compiler output data structures into JSON (or a similar language). Thus, no need to write or maintain new grammar rules or anything like if one was using TextMate.

([1] The source structure can be used for outline and code folding, although in RustDT it is only used for the outline. Code folding is not supported in the IDE itself - I never got around to it since I don't find that functionality is that important really - I never use it when working with Java for example)

My point with all this, to summarize, is that if Eclipse were to get built-in support for TextMate grammars, even as a library, I don't see how that would be of any significant help for the Eclipse IDEs for Go, Rust, D, etc.. Even if one was being written from scratch. The impact would be minimal.

On the other hand, having a common language server protocol (a protocol for an external tool providing layer 2 and layer 3 functionality), that would be of immense use. Which is why I'm excited about the VSCode protocol thing. It's even better if part of the Eclipse support for this language server can be put in an Eclipse-agnostic Java library. Which is exactly what https://github.com/TypeFox/ls-api aims to provide, so that looks very promising too. (I haven't had the time yet to look into it in more detail.)

But it seems clear to me we should reuse things as much as possible. That means:

* Re-use the VS-Code protocol. It looks somewhat comprehensive already, and has more than one implementations - both client/IDE-wise, and language-wise. Like Sven mentioned, if there are some shortcomings we can always add some Eclipse-specific extensions to the protocol (although ideally this would be merged back into the standard protocol).

* Re-use common Java code across other Java-based IDEs, not just the Eclipse IDE. (The Eclipse IDE project has traditionally been very bad at this - but that is a whole other discussion)

I think the next step as far as Eclipse is concerned would be to have a Proof of Concept / prototype of an editor/IDE support using a VSCode Language Server. But who will take ownership of this?

On 17 June 2016 at 12:06, Mickael Istria <mistria@xxxxxxxxxx> wrote:
Thanks Tom et al. for the details,

On 06/15/2016 01:42 PM, Tom Schindl wrote:
On to bug 496114. I would stay away of doing the implementation on
So you basically think that this should be yet-another-editor in the IDE? I believe that would be a fine solution only if it becomes the default text editor and associate itself with about everything.
Also, I actually believe that your concern about changing the TextEditor risking to cascade to all other editors are actually something we want. If you think at how hyperlink or error markers are implemented at the Platform level and not in each editor, it seems to me that it is the right way forward. The features shouldn't rely that much on which editor is open, having the right feature set open according to the file format or some other metadata seems actually more relevant to me.
Also, let's imagine we introduce generic extensions for completion or coloration, there is nothing that would prevent specialized editor like JDT or JSDT or XText to override this behavior (that's already what they do actually, they override the default behavior of text editor -currently no behavior for coloration/completion- by their own implementation). If we hook the extensions at the right place, then we can hope most existing specialized editors would remain unchanged.

Mickael Istria
Eclipse developer at JBoss, by Red Hat
My blog - My Tweets

ide-dev mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit


ide-dev mailing list
To change your delivery options, retrieve your password, or unsubscribe from this list, visit