Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [wtp-dev] JSDT Parser

On 14 Mar 2016, at 16:33, Doug Schaefer wrote:

Thanks Michael. I guess I'm still confused. Max mentioned that JSDT was
using Orion

I said it was based on the same tech that Orion used - not used Orion directly.

The new thing coming out of Orion that we could possibly use was by Steve's own
words: just a few hours old.

So we'll look at it and see what we can use or not and give Steve and orion team feedback.

, but I haven't seen much evidence of that. I highly encourage
you do that (and by extension my QML work does that) to leverage as much of
the expertise on the Orion project as possible, especially as they work to
make their bits more consumable by other IDEs.

As Gorkem points out - I hope Orion contributed their required changes upstream to
the respective projects since otherwise we'll just end up with a closed ecosystem
inside Orion.

But I definitely like the approach orion is taking about providing services - its exactly
what we need IMO. Now just need Eclipse to start being able to consume such things in more
than just JSDT - but thats for another thread ;)

/max

Doug.

On Mon, Mar 14, 2016 at 11:26 AM, Michael Rennie <Michael_Rennie@xxxxxxxxxx>
wrote:

In Orion, we have a modified version of Esprima (its a bit old now,
version 2.0), that has more tolerance in it than the stock Esprima does. It
might provide the level of recovery you are looking for.

Our recovery is done in a few ways:

  1. we catch and record all exceptions, to make it far less 'throwy'
  2. we perform node fill-ins when an exception would leave the AST is bad state
  3. in some cases we rewind and try again based on state / tokens / and line endings

Our version can be found here:

http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client._javascript_/web/esprima/esprima.js

The tests we run on it (to give an idea of the kinds of things we can
recover from):

http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/tree/bundles/org.eclipse.orion.client._javascript_/web/js-tests/_javascript_/esprimaTolerantTests.js

And lastly, the bug for us to upgrade to the latest version of Esprima:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=473765

Michael Rennie

----- Original message -----
From: Eugene Melekhov emvv@xxxxxxx
Sent by: wtp-dev-bounces@xxxxxxxxxxx
To: "General discussion of project-wide or architectural issues." <
wtp-dev@xxxxxxxxxxx>
Cc:
Subject: Re: [wtp-dev] JSDT Parser
Date: Sat, Mar 12, 2016 10:31 AM

Angelo,

Yes, sure, I used that option. The problem is that esprima actually throws
a lot of syntax exceptions:

grep -c throwUnexpectedToken esprima.js
56
grep -c tolerateUnexpectedToken esprima.js
29

That means that in 2/3 error cases it throws exception and tolerate only
1/3. Shift meanwhile doesn't throws exceptions
in "Early Error" cases like this 0=0, which is tolerated by esprima. I
have a suspicion that most of esprima
"tolerated errors" are from that "Early Error" category, which means that
in fact Shift behaves like esprima :-)

I thought that it would be interesting to compare shift and esprima
behavior on large enough set of erroneous code
fragments. It might happen that they are very close actually :-)

I'm not a big expert with esprima, but do you use tolerant option like

explained at http://esprima.org/doc/ ?

2016-03-12 10:25 GMT+01:00 Eugene Melekhov emvv@xxxxxxx:

Angelo,

One more thing about parser tolerance. esprima for example throws an

exception parsing this "relatively simple"

declarations:

//function foo(a, b {}

//function foo(a, b, c,) {}

//function foo(a, b, {} {}

Yes sure, but it seems that Shift doesn't support tolerant parser

which is very required for a JS editor. See
https://github.com/shapesecurity/shift-java/issues/93

--
Eugene Melekhov

wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe

from this list, visit

https://dev.eclipse.org/mailman/listinfo/wtp-dev

--
Eugene Melekhov

wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/wtp-dev

wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/wtp-dev

wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/wtp-dev

/max
http://about.me/maxandersen

Back to the top