Bug 511124 - Same file contents are being repeatedly requested by the tooling
Summary: Same file contents are being repeatedly requested by the tooling
Status: RESOLVED FIXED
Alias: None
Product: Orion (Archived)
Classification: ECD
Component: JS Tools (show other bugs)
Version: 13.0   Edit
Hardware: PC Windows 7
: P2 normal (vote)
Target Milestone: 14.0   Edit
Assignee: Curtis Windatt CLA
QA Contact:
URL:
Whiteboard: 2017-02-24
Keywords: performance
Depends on:
Blocks:
 
Reported: 2017-01-26 13:10 EST by Curtis Windatt CLA
Modified: 2017-02-13 14:37 EST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Curtis Windatt CLA 2017-01-26 13:10:42 EST
1) Have chrome record network XHR calls
2) Open a file with large dependency graph (server.js)
Result:
The same file contents are read multiple times and similar file name searches are being done.

Causes similar performance problems when doing "Show Problems".
Comment 1 Curtis Windatt CLA 2017-01-26 13:17:44 EST
There are two major components to this problem

1) When we resolve a dependency (AMD/Node/etc) to its full file name we get the file contents, but we never pass the contents into Tern, so Tern makes an additional call to lookup the file.

2) Every time we see a new dependency (AMD/Node/etc) we to a search to get the full file name and then immediately look up the file contents, even though we have looked up that file contents before.
Comment 2 Curtis Windatt CLA 2017-01-26 13:54:21 EST
(In reply to Curtis Windatt from comment #1)
> 1) When we resolve a dependency (AMD/Node/etc) to its full file name we get
> the file contents, but we never pass the contents into Tern, so Tern makes
> an additional call to lookup the file.

http://git.eclipse.org/c/orion/org.eclipse.orion.client.git/commit/?id=7778f850e4fb04f276c82cd979b566011cd6c827
Fixed this issue by passing the contents back in our resolver plugin.  This requires a modification to Tern code, but it is relatively simple now that all the dependency plugins use modules.js.
Comment 3 Curtis Windatt CLA 2017-01-26 17:17:16 EST
(In reply to Curtis Windatt from comment #1)
> 2) Every time we see a new dependency (AMD/Node/etc) we to a search to get
> the full file name and then immediately look up the file contents, even
> though we have looked up that file contents before.

This has much less of an impact of performance as (1) because after the read, if it is a known file, we don't kick off Tern analysis. Here are two potential solutions, but implementing them may not be worth the tradeoff.

a) Have a cache in javaScriptPlugin mapping file paths to their content.  I implemented this and it does remove the need to perform a file read.  However, the cache has to be fairly large, otherwise it gets a lot of cache misses because the list of dependencies is calculated before each file's contents are returned.

b) Modify the getFile operation in Tern to separate the search portion (changing logical shortname into longname path) from the fetching of file contents.  This is already happening in scriptResolver outside of Tern.  The drawback is this will require modifying Tern and will require two calls from the worker to the client (to save one call to the server).
Comment 4 Curtis Windatt CLA 2017-01-30 09:47:16 EST
I'm closing this as FIXED.  The major performance issue has been fixed, there are now way less reads happening (and way less work happening to process the read contents).  As I explained in the previous comment we can remove a few more reads in special cases, but the cost in memory or code complexity would be too high.