Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [wtp-dev] How to disable parse of the internal JSDT AST?

Hi Gorkem,

Many thanks for your answer!

At first I would like to speak about VSCode which is very very fast even with big files. Why it is so fast when file is edited? The answer is that it doesn't parse the file to build an AST for syntax coloration, folding, match bracket, etc.
So when you open a big js, typescript, json file in VSCode the editor display the content of the file without syntax coloration. The compute of the token (done with TextMate and cached) is done in background and when compute is finished, the editor is colorized. With Eclipse which uses tokens scanner we have not this problem, because it's very very fast even with big file. 

Indeed, I have created my own editor at https://github.com/angelozerr/typescript.java/blob/master/eclipse/ts.eclipse.ide.ui.editor/src/ts/eclipse/ide/ui/editor/internal/TypeScriptEditor.java and the very cool thing is that JSDT provides _javascript_TextTools that I can use to support syntax coloration. The good news is that it's very very fast. When I open big file like https://github.com/Microsoft/TypeScript/blob/master/lib/tsserver.js the colorization is done without freeze.

With JSDT 2.0.0 editor, the open of https://github.com/Microsoft/TypeScript/blob/master/lib/tsserver.js takes 30 seconds every time (not only the first time). So my conclusion is that when a file is opened in the editor, it must NOT parse the content of JS file to create an AST like JSDT does. The parse of the AST should be done in background.

But you will tell me that if we do that, we loose some feature like code folding which is based on AST (IJavaElement). How VSCode support folding without building an AST? The answer is that VSCode cheat a little:) It support folding by using indentation. So if you write:

---------------------------
function a() {
  var b;
}
---------------------------

The folding is done for function. But if you write 


---------------------------
function a() {
var b;
}
---------------------------

The folding is not done. I like this idea and I will implement it in my own editor.

So I have decided to implement my own editor but based on some JSDT features like syntax coloration by using _javascript_TextTools. I tell me if JSDT could provide a lightweight JavaScritpt editor which supports commons features like syntax coloration, folding, match bracket which don't need the use of the AST. 

The JavaEditor could extend this lightweight editor and build an AST (on background if possible). After that other project could use this lightweight _javascript_ editor.

In other words, I think _javascript_ editor should never parse the content of the opened file to build an AST in order to that when user opens a big files, the editor doesn't freeze. The AST should be done in background and completion should be available only when this AST is built.

Perhaps I'm missing something, but it's my first feeling. What do you think about this idea?

Regard's Angelo

2016-04-04 15:34 GMT+02:00 Gorkem Ercan <gorkem.ercan@xxxxxxxxx>:


On 4 Apr 2016, at 7:24, Angelo zerr wrote:

Hi max,

many thanks for your answer.

1) double parsing problem (one parse on JSDT client side and a second parse
on tern server/tsserver). I would like to avoid the parse on JSDT client
side

why not work on reusing the AST already available on client side and pass
it to server side ?

To do that we need a common _javascript_ Engine. For ternjs, node.js -> J2V8
-> Nashorn for performance.

I think a common AST model is sufficient for reducing parsing.
EsprimaParser(our java wrapper around esprima.js) actually has the ESTree model
that tern.js uses, it discards it at the moment after converting to DOM AST model.
I think we can hold it (as JSON) to pass to tools like tern or eslint
which uses ESTree AST model.

I think the real problem is, at least at the time I have looked at it,
tsserver does not work with AST’s but files directly.

So we could use J2V8, but it will require that we must implement some
node.js module like "fs" which are consumed in some tern plugins for some
features (like completion for required module)

For TypeScript, in my case I use tsserver which is linked to node.js. I
could implement my own Language Service Host (as TypEcs, and Eclipse
TypeScript have done), but I think it's a bad idea:

 * VSCode works with tsserver and tsserver manage a lot of logic like the
use of tsconfig.json
 * tsserver is available when you install TypeScript, so with
typescript.java, it's very easy to change version of TypeScript
(with TypEcs, and Eclipse TypeScript, you need to recompile the plugins,
and you are not sure that it works) .


Thought that was one of the advantages we could do know by using esprima
to parse client side.

2) peformance problem with big file which can take 30 secs (see
https://bugs.eclipse.org/bugs/show_bug.cgi?id=490898)

That one does look nasty.


Yes I know -(


Is this with a cold parse. We already know that nashorn is slow on the first few executions
until JVM/JIT optimizes.


3) disable validation of the WTP Validator because Esprima throw errors
with TypeScript syntax (ex: var s: string) and tsserver is able to validate
js, ts files.t).

If this internal AST cannot be disable (with an extension point?), I tell
me if I must implement my own editor (I would like to avoid doing that).

Since you are doing TypeScript I think you'll need to *anyway* as long as
Eclipse
don't have a generic editor fwk to extend. But Gorkem and guys knows
better what
is feasible.

You mean that I should implement my own TextEditor?


I am looking at the enhancement requests you have filed for typescript.

490429 [Mark occurences] override mark occurrence
490427 [formatter] CodeFormatter extension point
490426 [outline] Override content outline
485321 Extension Point to delegate refactoring
479807 More extension for Find References (Ctrl+Shift+G)
470328 Extension point to delegate JSDoc "Generate Element Comment"
482888 Disable JSDT Hyperlink with extension point
482971 Disable some javaCompletionProposalComputer with a new extension point


Looking at the list I am actually having trouble finding the features that you want to
use from JSDT Editor. Perhaps implementing a new editor is the way to proceed.



Regard's Angelo

/max
http://about.me/maxandersen

_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/wtp-dev

_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/wtp-dev
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/wtp-dev


Back to the top