[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [egit-dev] Expected performance profile of staging changes and committing then

Hi Markus,

On Tue, Nov 6, 2012 at 12:14 PM, Markus Duft <markus.duft@xxxxxxxxxx> wrote:
> On 11/06/2012 10:14 AM, Tomasz Zarna wrote:
>>> Both jobs are triggered by the IndexChangedEvent i guess (or at least the re-indexing).
>>> This event is IMHO just too stupid, as it carries no information about /what/ changed.
>>
>> Out of curiosity, is there a bug for that?
>
> Ofc, there is the very long standing JGit bug [1]. It is there for over a minute now, so i wonder why nobody fixed it yet :D
>
> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=393642

Thanks for the detailed information and for raising that bug.
Hopefully someone can look into properly fixing this.

I know little about the internals of {j,e}git but I'd be more than
happy to test any proposed fixes on my repository, add profiler traces
and whatever else is needed.

Robert


>
>>
>> Cheers,
>> Tomasz
>>
>> On Tue, Nov 6, 2012 at 8:15 AM, Markus Duft <markus.duft@xxxxxxxxxx <mailto:markus.duft@xxxxxxxxxx>> wrote:
>>
>>     On 11/05/2012 05:38 PM, Robert Munteanu wrote:
>>     > Hi,
>>     >
>>     [snip]
>>     > Some background information: my repository is pretty large, 17000
>>     > files, 9000 directories, 1.2GB total size, 2.5 MB for .git/index. My
>>     > computer is reasonably fast ( SSD, i7, Linux ) so I wouldn't look
>>     > there for performance problems.
>>
>>     Hi!
>>
>>     We have a similar repo - ~30000 files, 1.1GB total, 5.5MB index. I use a linux workstation with 16 cores, 12G RAM and a spinning 10k rpm disk.
>>
>>     >
>>     > I don't use the commit dialog since the performance is too slow [1]
>>     > so I fell back to using the staging view.
>>
>>     Me too - i already contributed some changes to EGit vastly increasing the performance of the staging view, as it was too slow for me too ;)
>>
>>     >
>>     > When I use the staging view I observe the following:
>>     >
>>     > 1. Staging changes involves at least two background jobs:
>>     >
>>     > - Git repository refresh
>>     > - Re-indexing repository $NAME
>>
>>     Both jobs are triggered by the IndexChangedEvent i guess (or at least the re-indexing). This event is IMHO just too stupid, as it carries no information about /what/ changed, i just tells that /something/ in your 2.5MB index changed, thus EGit has to recalculate everything. This is an issue for me too, as you can imagine, as our index is even bigger :/
>>
>>     >
>>     > 2. Committing changes involves at least three background jobs
>>     >
>>     > - Committing changes ( which is pretty fast )
>>     > - Re-indexing repository $NAME
>>     > - Building commit info
>>
>>     Same here, the re-indexing is triggered by the index change. I profiled those jobs a lot, and i guess for what they do, they are pretty fast. The problem is what they do - they are forced to do much to much, as the information about the actual changes is missing.
>>
>>     >
>>     > Both these operations take a lot of time and I would expect them to be
>>     > instant, since I'm changing 2 files out of 17.000 and egit knows which
>>     > files have changed and doesn't need to recalculate the whole
>>     > repository state.
>>
>>     This describes what we all would like it to be :D
>>
>>     Regards,
>>     Markus
>>
>>     >
>>     > Am I missing out on something? I've captured yourkit snapshots of both
>>     > the 'stage' and 'commit' operations and I can provide them if needed.
>>     > Before that I'd like to know whether this is expected and I should
>>     > stop worrying and start drinking more tea or considered a performance
>>     > bug and should be entered in bugzilla .
>>     >
>>     > Thanks,
>>     >
>>     > Robert
>>     >
>>     >
>>     > [1]: 331273: Commit operations opens and reads almost all files
>>     > https://bugs.eclipse.org/bugs/show_bug.cgi?id=331273
>>     >
>>     > --
>>     > Sent from my (old) computer
>>     > _______________________________________________
>>     > egit-dev mailing list
>>     > egit-dev@xxxxxxxxxxx <mailto:egit-dev@xxxxxxxxxxx>
>>     > https://dev.eclipse.org/mailman/listinfo/egit-dev
>>     >
>>     _______________________________________________
>>     egit-dev mailing list
>>     egit-dev@xxxxxxxxxxx <mailto:egit-dev@xxxxxxxxxxx>
>>     https://dev.eclipse.org/mailman/listinfo/egit-dev
>>
>>
> _______________________________________________
> egit-dev mailing list
> egit-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/egit-dev



--
Sent from my (old) computer