Re: [egit-dev] Expected performance profile of staging changes and commi

[Markus Duft <markus.duft@xxxxxxxxxx>]

On 11/05/2012 05:38 PM, Robert Munteanu wrote:
> Hi,
> 
[snip]
> Some background information: my repository is pretty large, 17000
> files, 9000 directories, 1.2GB total size, 2.5 MB for .git/index. My
> computer is reasonably fast ( SSD, i7, Linux ) so I wouldn't look
> there for performance problems.

Hi!

We have a similar repo - ~30000 files, 1.1GB total, 5.5MB index. I use a linux workstation with 16 cores, 12G RAM and a spinning 10k rpm disk.

> 
> I don't use the commit dialog since the performance is too slow [1]
> so I fell back to using the staging view.

Me too - i already contributed some changes to EGit vastly increasing the performance of the staging view, as it was too slow for me too ;)

> 
> When I use the staging view I observe the following:
> 
> 1. Staging changes involves at least two background jobs:
> 
> - Git repository refresh
> - Re-indexing repository $NAME

Both jobs are triggered by the IndexChangedEvent i guess (or at least the re-indexing). This event is IMHO just too stupid, as it carries no information about /what/ changed, i just tells that /something/ in your 2.5MB index changed, thus EGit has to recalculate everything. This is an issue for me too, as you can imagine, as our index is even bigger :/

> 
> 2. Committing changes involves at least three background jobs
> 
> - Committing changes ( which is pretty fast )
> - Re-indexing repository $NAME
> - Building commit info

Same here, the re-indexing is triggered by the index change. I profiled those jobs a lot, and i guess for what they do, they are pretty fast. The problem is what they do - they are forced to do much to much, as the information about the actual changes is missing.

> 
> Both these operations take a lot of time and I would expect them to be
> instant, since I'm changing 2 files out of 17.000 and egit knows which
> files have changed and doesn't need to recalculate the whole
> repository state.

This describes what we all would like it to be :D

Regards,
Markus

> 
> Am I missing out on something? I've captured yourkit snapshots of both
> the 'stage' and 'commit' operations and I can provide them if needed.
> Before that I'd like to know whether this is expected and I should
> stop worrying and start drinking more tea or considered a performance
> bug and should be entered in bugzilla .
> 
> Thanks,
> 
> Robert
> 
> 
> [1]: 331273: Commit operations opens and reads almost all files
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=331273
> 
> --
> Sent from my (old) computer
> _______________________________________________
> egit-dev mailing list
> egit-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/egit-dev
>