Re: [egit-dev] Expected performance profile of staging changes and commi

[Tomasz Zarna <tzarna@xxxxxxxxx>]

> Both jobs are triggered by the IndexChangedEvent i guess (or at least 
the re-indexing). 
> This event is IMHO just too stupid, as it carries no 
information about /what/ changed.

Out of curiosity, is there a bug for that?

Cheers,
Tomasz

On Tue, Nov 6, 2012 at 8:15 AM, Markus Duft <markus.duft@xxxxxxxxxx> wrote:

On 11/05/2012 05:38 PM, Robert Munteanu wrote:
> Hi,
>
[snip]

> Some background information: my repository is pretty large, 17000
> files, 9000 directories, 1.2GB total size, 2.5 MB for .git/index. My
> computer is reasonably fast ( SSD, i7, Linux ) so I wouldn't look
> there for performance problems.

Hi!

We have a similar repo - ~30000 files, 1.1GB total, 5.5MB index. I use a linux workstation with 16 cores, 12G RAM and a spinning 10k rpm disk.

>
> I don't use the commit dialog since the performance is too slow [1]
> so I fell back to using the staging view.

Me too - i already contributed some changes to EGit vastly increasing the performance of the staging view, as it was too slow for me too ;)

>
> When I use the staging view I observe the following:
>
> 1. Staging changes involves at least two background jobs:
>
> - Git repository refresh
> - Re-indexing repository $NAME

Both jobs are triggered by the IndexChangedEvent i guess (or at least the re-indexing). This event is IMHO just too stupid, as it carries no information about /what/ changed, i just tells that /something/ in your 2.5MB index changed, thus EGit has to recalculate everything. This is an issue for me too, as you can imagine, as our index is even bigger :/

>
> 2. Committing changes involves at least three background jobs
>
> - Committing changes ( which is pretty fast )
> - Re-indexing repository $NAME
> - Building commit info

Same here, the re-indexing is triggered by the index change. I profiled those jobs a lot, and i guess for what they do, they are pretty fast. The problem is what they do - they are forced to do much to much, as the information about the actual changes is missing.

>
> Both these operations take a lot of time and I would expect them to be
> instant, since I'm changing 2 files out of 17.000 and egit knows which
> files have changed and doesn't need to recalculate the whole
> repository state.

This describes what we all would like it to be :D

Regards,
Markus

>
> Am I missing out on something? I've captured yourkit snapshots of both
> the 'stage' and 'commit' operations and I can provide them if needed.
> Before that I'd like to know whether this is expected and I should
> stop worrying and start drinking more tea or considered a performance
> bug and should be entered in bugzilla .
>
> Thanks,
>
> Robert
>
>
> [1]: 331273: Commit operations opens and reads almost all files
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=331273
>
> --
> Sent from my (old) computer
> _______________________________________________
> egit-dev mailing list
> egit-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/egit-dev
>
_______________________________________________
egit-dev mailing list
egit-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/egit-dev