Community
Participate
Working Groups
A post on the mailing list brought up the idea of using Tern to provide inferred content assist proposals in Orion: http://dev.eclipse.org/mhonarc/lists/orion-dev/msg02948.html Here is a link to the Tern demo for content assist: http://ternjs.net/doc/demo.html Assuming Tern works right out of the box, I can see few immediate benefits for Orion: 1. Can be run in a worker, which would improve the performance of our content assist 2. Would prevent us from having to use modified indexes for our lib extensions (huge win) 3. Appears to provide decent formatting right out of the box 4. Would (likely) completely replace our need for our own inferencing / env 5. Can be used (in the future) to provide other features like jump to decl (think Ctrl+Click in Eclipse), JSdoc hovers, etc Tern does rely on the Acorn parser, which we do not yet use. We should be able to remove this need and have Tern use our shared AST since both Acorn and Esprima create MDN-compliant ASTs.
Many thank's Michael to have created this bug. My POC about tern & orion can be found at https://github.com/angelozerr/tern.orion Completion with tern starts working. >1. Can be run in a worker, which would improve the performance of our content >assist The Tern CodeMirror Addon can be used with worker too. I will do the same thing for orion & tern. >5. Can be used (in the future) to provide other features like jump to decl (think >Ctrl+Click in Eclipse), JSdoc hovers, etc I would like to support that, but my problem is about Orion API which seems not support : 1) custom Text Hover 2) custom Hyperlink with Ctrl+Clic Note that tern is able to manage refactoring to rename variable. Tern provides several cool plugin like : * jQuery => see https://github.com/angelozerr/tern.java/wiki/Tern-&-jQuery-support * Node.js => https://github.com/angelozerr/tern.java/wiki/Tern-&-Node.js-support * Angular ... It's not very complicated to develop tern plugin. we have done that in our project (we integrate Tern in Eclipse IDE) for YUI, Grunt, cordova. See at https://github.com/angelozerr/tern.java/tree/master/core/tern.server.nodejs/node_modules/tern/plugin Regards Angelo
For text hover, bug was created last year https://bugs.eclipse.org/bugs/show_bug.cgi?id=373456 But Orion seems to manage hover for TODO annotation ,error annotation. DO you think it's possible to manage custom hover by creating annotation?
(In reply to Angelo ZERR from comment #2) > For text hover, bug was created last year > https://bugs.eclipse.org/bugs/show_bug.cgi?id=373456 > > But Orion seems to manage hover for TODO annotation ,error annotation. DO > you think it's possible to manage custom hover by creating annotation? Thank for the pointers Angelo, I will have a look at your integrations. As far as using annotations / hovers, I think Silenio would have more information for you. > Note that tern is able to manage refactoring to rename variable. Yes, refactoring is something that we are sorely lacking at the moment.
anything happend in this area/investigation ?
(In reply to Max Rydahl Andersen from comment #4) > anything happend in this area/investigation ? To a degree: 1. we are currently working on the support to access other files from the current file being edited (Jump to decl type access) 2. I tried to get Tern working nicely with Esprima (the parser base for all of our tools), but there were some rough edges (ie. not working right) 3. I also tried to get all of our tooling working using the Acorn parser - which would allow us to just use Tern as-is, and I managed to get to the point that all of our parser unit tests were passing, but there was some troubles with ESLint (the linter used by Orion) Something I did not try was to use Tern + Acorn as-is for nav + additional assist and use Esprima for parsing + linting + tool support. The reason being that we do not want two ASTs hanging around, and while both parsers do generate a Mozilla-compliant AST there are subtle differences that the respective tooling does not tolerate. For example, Esprima adds a comments array + attaches comments to the relevant nodes, which is used by ESLint and other tools. Acorn does not handle comments at all except for a callback while parsing and its tolerant mode fills in the AST with a kind of recovered node that ESLint and other tools do not tolerate.
(In reply to Michael Rennie from comment #5) > 2. I tried to get Tern working nicely with Esprima (the parser base for all > of our tools), but there were some rough edges (ie. not working right) I'm going to take another swing at this type of integration, that way (once it is working) we get the power of Tern but can still use all of our Esprima-based tools as-is (with no additional ASTs being created, etc.)
> I'm going to take another swing at this type of integration, that way (once it is working) we get the power of Tern but can still use all of our Esprima-based tools as-is (with no additional ASTs being created, etc.) It should be very fantastic. Today for validation, I have started to implement a lint tern plugin https://github.com/angelozerr/tern.lint You can see validation rules at https://github.com/angelozerr/tern.lint/wiki/Validation-Rules It should be very cool if I can integrate too ESLint inside tern.java
> For example, Esprima adds a comments array + attaches comments to the relevant nodes, which is used by ESLint and other tools. Acorn does not handle comments at all except for a callback I'm not sure, but it seems that there is an PR for that https://github.com/marijnh/acorn/pull/116
Removing the milestone. While I did get Tern up and running on esprima, etc, I did not have a chance to really try out everything Tern can do - I really only got as far as seeing if it could what our env does, especially around treatment of JSDoc while inferencing. A couple of things I found are that: 1. Tern consistently selects the wrong doc for a node when > 1 doc node is present 2. regardless of what you specify in doc, Tern will ignore it if the type does not (yet) exist in the server
Hi Michael, > 1. Tern consistently selects the wrong doc for a node when > 1 doc node is present > 2. regardless of what you specify in doc, Tern will ignore it if the type does not (yet) exist in the server It seems that you have some troubles with tern. Have you create tern issues for that? I think integration of tern.js inside Orion should be very cool. There is more and more JS framworks managed by tern like google-closure, node express, mongodb, etc. tern.js can be used inside Eclipse IDE too. I think having a commons JS inference engine is very good thing (Web editor orion and Eclipse IDE). The tern community is growing (Mozilla has integrated tern)... Hope one day tern.js could be used really inside orion.
(In reply to Angelo ZERR from comment #10) > Hi Michael, > > > 1. Tern consistently selects the wrong doc for a node when > 1 doc node is present > > 2. regardless of what you specify in doc, Tern will ignore it if the type does not (yet) exist in the server > > It seems that you have some troubles with tern. Have you create tern issues > for that? https://github.com/marijnh/tern/issues/392 https://github.com/marijnh/tern/issues/391 > > I think integration of tern.js inside Orion should be very cool. I agree. Since my Tern knowledge is not nearly as good as yours, would you be interested in helping to integrate Tern?
> would you be interested in helping to integrate Tern? I'm very busy with integration of tern.js with tern.java (https://github.com/angelozerr/tern.java) to use tern.js inside Eclipse IDE (and AngularJS Eclipse), so I'm afraid that it will hard for me to help you a lot (more I have never played with Orion). I think to integrate tern inside orion we should start with https://github.com/angelozerr/tern.orion tern.orion is a glue between tern and orion completion. The tern server is created on client side (Web Browser). It works for just a JS file, but in a JS project you have a lot of JS. To support that, tern server must be created on server side, in other words tern should be executed with node.js and tern orion completion comunicate with the node.js server which use tern server. So tern.orion should support tern : * on client side (like today) : the tern server is created with Web Browser. It's usefull for a simple editor. * on server side : the tern server is created with node.js. In this case we can have a lot of JS files. Once "server side" will be implemented, I think it will be easy to integrate tern inside orion.
(In reply to Angelo ZERR from comment #12) > > would you be interested in helping to integrate Tern? > > I'm very busy with integration of tern.js with tern.java > (https://github.com/angelozerr/tern.java) to use tern.js inside Eclipse IDE > (and AngularJS Eclipse), so I'm afraid that it will hard for me to help you > a lot (more I have never played with Orion). > > I think to integrate tern inside orion we should start with > https://github.com/angelozerr/tern.orion > > tern.orion is a glue between tern and orion completion. The tern server is > created on client side (Web Browser). It works for just a JS file, but in a > JS project you have a lot of JS. To support that, tern server must be > created on server side, in other words tern should be executed with node.js > and tern orion completion comunicate with the node.js server which use tern > server. > > So tern.orion should support tern : > > * on client side (like today) : the tern server is created with Web > Browser. It's usefull for a simple editor. > * on server side : the tern server is created with node.js. In this case we > can have a lot of JS files. > > Once "server side" will be implemented, I think it will be easy to integrate > tern inside orion. Thanks for the suggestions Angelo. I opened CQs for Tern: https://dev.eclipse.org/ipzilla/show_bug.cgi?id=8827 and Acorn: https://dev.eclipse.org/ipzilla/show_bug.cgi?id=8826
*** Bug 426246 has been marked as a duplicate of this bug. ***
Created Bug 456737 - [JSDT] Question about applying Tern and Esprima
Hi Michael, I would like to know if you are working again about acorn + ESLint integration. Do you know if it's possible to use together even if some ESLInt rules doesn't work (if I understood, acorn doesn't store some comments that ESLint requires). If it's possible to use acorn + ESLint, I will try to integrate inside tern.java. Thank's for your information. Regard's Angelo
(In reply to Angelo ZERR from comment #16) > Hi Michael, > > I would like to know if you are working again about acorn + ESLint > integration. > > Do you know if it's possible to use together even if some ESLInt rules > doesn't work (if I understood, acorn doesn't store some comments that ESLint > requires). > > If it's possible to use acorn + ESLint, I will try to integrate inside > tern.java. > > Thank's for your information. > > Regard's Angelo I have been working on it. Have a look at the my topic branch: http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/log/?h=mrennie/acorn_parser. The ESLint work is in /javascript/eslint/lib. The rough spots (so far) are: 1. the shape of Acorn's tokens are not the same as Esprima, so any rule that uses tokens will have to be modified to use the Acorn API for comparing tokens: Acorn.tokTypes. The other option is that you could provide a function callback to the parser option 'onToken' and 'convert' acorn tokens to Esprima-style before you save them. Something like: var ast = Acorn.parse(text, { ranges: true, locations: true, onToken: function(token) { tokens.push(_convertToEsprima(token)); }, onComment: comments, }); 2. Tokens and comments are not collected / attached by default, so you have to do something like: var tokens = []; var comments = []; var ast = Acorn.parse(text, { ranges: true, locations: true, onToken: tokens, onComment: comments, }); ast.source = text; ast.comments = comments; ast.tokens = tokens; Estraverse.attachComments(ast, comments, tokens); when parsing to get an AST that is 'Esprima-like' and will work (almost 100%) with ESLint. 3. The tolerant parser (acorn_loose), does not always produce the same AST output for valid JavaScript as the normal (acorn) parser does. See for example https://github.com/marijnh/acorn/issues/199. 4. When you do have to parse invalid source the tolerant parser inserts 'dummy' nodes to fill in the tree, these nodes are of type 'Identifier' and will be reported as unused by ESLint. The only workaround I have for that is to modify ESLint to ignore them. IMO it would be better if the 'dummy' node had a non-standard type id that would not be recognized by tools, something like 'RecoveredNode', etc. Hope this helps.
Thanks Michael for your feedback and for working about acorn/eslint integration. It seems that today it's not really easy to use acorn/eslint together without changing some code. I'm waiting for that you finish your work to integrate it to tern.java Please tell me when you think it will be possible (without changing some acorn or eslint code at hand).
Pushed the branch for tern integration: http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/log/?h=tern0_6_2 The first commit only contains the 0.6.2 version of Tern and none of the integration work has been done.
> Pushed the branch for tern integration: Very cool news! In tern.java case, I'm integrating serveral linter like ESLInt, JSHint, etc. I have developped several tern plugins that perhaps Orion could be interested like a fork of angular (to support Angular Expression), compeltion guess https://github.com/angelozerr/tern.java/wiki/Tern-&-Completion-Guess-support, etc You can see the whole tern plugins at https://github.com/angelozerr/tern.java/wiki/Getting-Started#tern-modules Don't hesitate to contact me if you need more information.
Ok, with the last few dozen commits, Tern is now running as our default content assist provider. I am going to close this bug and open separate ones for the remaining work, which includes: 1. get multi-file assist working properly - currently this is disabled because of sync vs. async operations and the problems that come along with that (bug 464821) 2. our old index files have to made into their own tern plugins (the stubs are in master, but not enabled yet) (bug 464822) 3. be able to load plugins on the fly, based on the prefs and the eslint-env directives (bug 464823) 4. provide a pref page to get / install / remove / disable tern plugins in your workspace (bug 464824)
