Community
Participate
Working Groups
When linking with a project from github.com I sometimes get a timeout exception because the file store for git are so slow on the server. Need to figure out how to improve this. A way to measure all stuff going on in the server would be nice too.
JohnA: One of the reasons it is slow is that it goes back to the remote git server too often. For example on each call to fetchInfo() it does a git pull for that file. How this should behave really depends on the workflow. If the user knows they are the only author of the remote repository, then we can just always use the local cache on the Eclipse Web server. If there are other users or clients pushing to the same github, then we need to be able to synchronize with them.
JohnA: One small step that comes to mind: Within a single GET, we should be able to satisfy it with a single request to the git repository. For example if I GET on a directory today, it appears to perform a separate git pull for each member of that directory. If it could do a single recursive pull on the directory rather than on each child, it would ensure there is at most one round trip to the git server for each round trip between browser and EW server.
Created attachment 186625 [details] Patch v01 I'm attaching a patch: a performance test for the most common operations for git file store. I hope it will serve as a reference point. I will now implement what John suggested in comment 2 and see if the test will indicate any improvement. Of course, John's idea makes totally sense to me, I would like to see if the perf. test is any good :)
Created attachment 186626 [details] Patch v02 Another patch: this one adds a new performance test which manipulates the workspace with HTTP requests, so any perf. improvement in o.e.e4.webide.server.servlets should be noticed. Speaking of which, the patch also modifies the way DirectoryHandlerV1 list directory children and makes GitFileStore aware of the EFS.CACHE flag. These simple changes resulted in a noticeable perf gain (-15% in CPU time, -28% in elapsed process).
I ran the test from Patch v01 for both git filestore and local file system. The results are devastating, git fs is over 100x slower then local fs (see bug 334285, comment 1).
Created attachment 187124 [details] Another fix This fix gives ca 7% improvement when setting file contents. This operation turned out to be a hot spot, but no surprise here: after setting a new content git add, commit, pull and push are called. The patch contains some extra tests as well. Another possible fix here could be to divide pull into fetch and merge, and do the later only if fetch (for the current branch) resulted in any changes being available.
Created attachment 187125 [details] mylyn/context/zip
This should be no longer an issue once git client (bug 336116) and git REST API are in place.
GitFileStore is no more part of the Orion server.
