Bug 407174 - Git log performance very slow
Summary: Git log performance very slow
Status: RESOLVED FIXED
Alias: None
Product: Orion (Archived)
Classification: ECD
Component: Git (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: 3.0 RC2   Edit
Assignee: Gabriel Luong CLA
QA Contact:
URL:
Whiteboard:
Keywords: helpwanted, performance
Depends on:
Blocks:
 
Reported: 2013-05-03 10:10 EDT by John Arthorne CLA
Modified: 2013-06-14 07:59 EDT (History)
2 users (show)

See Also:
Szymon.Brandys: review+


Attachments
Profile output shows the nature of the problem (24.47 KB, image/png)
2013-05-03 10:13 EDT, John Arthorne CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description John Arthorne CLA 2013-05-03 10:10:33 EDT
I sometimes see test failures in GitLogTest where the cause is "operation took too long". I noticed when running this that is typically takes 10-12 seconds to execute the log command in this test on my machine. I ran the test in the profiler, and found almost all the time is in Commit.getTags(). This operation walks over every tag in the repository and compares commit id to the one we are processing. In a Git Log operation, we do this for 50 commits at at time, which means walking the entire tag list 50 times. We could just walk the repository once and hold onto the result. This is what we do for branches.. see Commit.get/setCommitToBranchMap and how it is used in Log.getChildren().
Comment 1 John Arthorne CLA 2013-05-03 10:11:45 EDT
Marking as a candidate for 3.0. I might have time to get to this, but if not it's a good simple bug for someone getting started.
Comment 2 John Arthorne CLA 2013-05-03 10:13:57 EDT
Created attachment 230459 [details]
Profile output shows the nature of the problem
Comment 3 Gabriel Luong CLA 2013-05-27 17:20:55 EDT
https://orion.eclipse.org/git/reviewRequest.html#https://github.com/gabrielluong/orion.server.git_89eb7fb6b41c05b7d21435441e8fb40350e5ada6

Added check for when commitToTagMap does not contain the commit id
https://orion.eclipse.org/git/reviewRequest.html#https://github.com/gabrielluong/orion.server.git_99edb06d92c0f7c7010cd7c114f8cde61bfe107c

I assert that I authored 100% of the content of this contribution and have the rights to donate the content to Eclipse under the EPL
Comment 4 Gabriel Luong CLA 2013-05-28 10:21:16 EDT
Added my github commit just in case the process is a bit different for orion.server

Added a commitToTagMap similar to commitToBranchMap, which only fetches the map once and is set to every commit. Perhaps it should only pass in the tags relevant to the commit (getCommitToTagMap().get(revCommit.getId()) instead of passing the entire map? 
https://github.com/gabrielluong/orion.server/commit/89eb7fb6b41c05b7d21435441e8fb40350e5ada6

Added a check for npe when the revCommit id is not in the map.
https://github.com/gabrielluong/orion.server/commit/99edb06d92c0f7c7010cd7c114f8cde61bfe107c
Comment 5 John Arthorne CLA 2013-05-28 10:25:27 EDT
Szymon can you review this, I'm completely swamped.
Comment 6 Gabriel Luong CLA 2013-06-06 14:11:44 EDT
Review bump
Comment 7 Gabriel Luong CLA 2013-06-13 11:42:09 EDT
Review bump!
Comment 8 Szymon Brandys CLA 2013-06-14 07:59:05 EDT
Looks good, tests pass, merged.