Bug 490898 - JSDT 2.0.0 very slow with big file
Summary: JSDT 2.0.0 very slow with big file
Status: NEW
Alias: None
Product: JSDT
Classification: WebTools
Component: General (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Victor Rubezhny CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-04-01 08:37 EDT by Angelo ZERR CLA
Modified: 2016-04-07 10:35 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Angelo ZERR CLA 2016-04-01 08:37:40 EDT
It seems that the parse with Nashorm+Esprima takes a lot of times with big file. 

I have tried to open the big JS file https://github.com/Microsoft/TypeScript/blob/master/lib/tsserver.js with the JSDT 2.0.0 Editor (getted source code from the git master) and it takes between 20-30 seconds (multiply with 2 because it is parsed twice) although with JSDT 1.x it takes less 2 seconds.

I have tested with Windows 64 bits + jre1.8.0_77. I have copied/pasted nashorn.jar in the org.eclipse.wst.jsdt.core project (is it the problem?).

If you need more info, don't hesitate to tell me.
Comment 1 Eugene Melekhov CLA 2016-04-07 07:32:36 EDT
tssserver.js is about 2.5 Mb.
The resulting parsed JSON is 33Mb, so you can imagine while it takes so long to parse and then convert to the dom model.

Of course it could be done asynchronously, but anyway this Java->JS->Java->JS conversion seems to be not a very good idea. And that's why using Google closure compiler looks like better solution.
Comment 2 Angelo ZERR CLA 2016-04-07 07:52:48 EDT
Thank's Eugene for your feedack.

Have you tested with Google closure compiler? Which times take it?
Comment 3 Gorkem Ercan CLA 2016-04-07 09:08:09 EDT
(In reply to Eugene Melekhov from comment #1)
> tssserver.js is about 2.5 Mb.
> The resulting parsed JSON is 33Mb, so you can imagine while it takes so long
> to parse and then convert to the dom model.
> 

In our current nashorn based solution Esprima model is not converted to JSON we basically traverse java representation of the esprima parser. I have done some testing with vanilla esprima and it actually takes time to parse the file. I suspect conversion to DOM AST is less of a problem in this case.
Comment 4 Eugene Melekhov CLA 2016-04-07 10:35:37 EDT
(In reply to Gorkem Ercan from comment #3)
> In our current nashorn based solution Esprima model is not converted to JSON
> we basically traverse java representation of the esprima parser.
It's java representation of JS object with all that string property names etc and traversing it with string comparison etc really takes time.

Here are results for opening and closing that file in JSDT editor 3 times
First number is parsing time in seconds
Second number is translation time in seconds

Parsing finished 10.738  7.854
Parsing finished 18.736  8.316
Parsing finished 12.929  5.261
-------
Parsing finished  6.194  4.826
Parsing finished  6.036  3.830
-------
Parsing finished  4.562  4.982
Parsing finished  4.675  3.942

Of course first run is slower, others are better. Funny though that file is parsed 3 times at first run and twice at subsequent ones.

> I have done
> some testing with vanilla esprima and it actually takes time to parse the
> file. I suspect conversion to DOM AST is less of a problem in this case.

And as you can see conversion takes almost the same time as parsing, which seems "not right" :-)