[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [egit-dev] Expected performance profile of staging changes and committing then
|
On 11/06/2012 04:18 PM, Robert Munteanu wrote:
> Hi Markus,
>
> On Tue, Nov 6, 2012 at 12:14 PM, Markus Duft <markus.duft@xxxxxxxxxx> wrote:
>> On 11/06/2012 10:14 AM, Tomasz Zarna wrote:
[snip]
>> [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=393642
>
> Thanks for the detailed information and for raising that bug.
> Hopefully someone can look into properly fixing this.
you might be interested in these changes i just pushed:
[1] https://git.eclipse.org/r/#/c/8535/
[2] https://git.eclipse.org/r/#/c/8536/
they belong together and bring staging/unstaging time down to a millisecond for me :D ofc this is still kind of a (dirty?) hack ;) however all unit tests run through with it and i did not observe any obvious problems with it. unit tests run a little slower since the Repository constructor now already causes an index read, but i don't think this will do harm in real-world scenarios as Repository is not a thing that is constructed all the time from scratch. Experts may now prove me wrong ;)
For still unknown reason events don't exactly behave as i'd expect them sometimes. it happened that the code did not detect changes from one to the other index state. i was not really able to reproduce this case, but still... i don't know exactly yet. in such a case the refresh button in the staging view would trigger a full refresh. Sometimes a full refresh is triggered too, which i don't fully understand either, but most of the time it's fast ;)
The initial construction of the IndexDiffCacheEntry still does a full scan, so does a RefChangedEvent (caused by every ref update (commit, rebase, ...)).
>
> I know little about the internals of {j,e}git but I'd be more than
> happy to test any proposed fixes on my repository, add profiler traces
> and whatever else is needed.
Would be cool if you can build it yourself. if you can't, [3] is a pre-built snapshot with my change (and all (except one) changes from [4]).
[3] http://dev.gentoo.org/~mduft/wamas-egit.tgz
[4] https://git.eclipse.org/r/#/q/status:open+AND+owner:mduft%2540gentoo.org,n,z
(it's an update site archive, so you should be able to just install from it into your eclipse ;))
Markus
>
> Robert
>
>
>>
>>>
>>> Cheers,
>>> Tomasz
>>>
>>> On Tue, Nov 6, 2012 at 8:15 AM, Markus Duft <markus.duft@xxxxxxxxxx <mailto:markus.duft@xxxxxxxxxx>> wrote:
>>>
>>> On 11/05/2012 05:38 PM, Robert Munteanu wrote:
>>> > Hi,
>>> >
>>> [snip]
>>> > Some background information: my repository is pretty large, 17000
>>> > files, 9000 directories, 1.2GB total size, 2.5 MB for .git/index. My
>>> > computer is reasonably fast ( SSD, i7, Linux ) so I wouldn't look
>>> > there for performance problems.
>>>
>>> Hi!
>>>
>>> We have a similar repo - ~30000 files, 1.1GB total, 5.5MB index. I use a linux workstation with 16 cores, 12G RAM and a spinning 10k rpm disk.
>>>
>>> >
>>> > I don't use the commit dialog since the performance is too slow [1]
>>> > so I fell back to using the staging view.
>>>
>>> Me too - i already contributed some changes to EGit vastly increasing the performance of the staging view, as it was too slow for me too ;)
>>>
>>> >
>>> > When I use the staging view I observe the following:
>>> >
>>> > 1. Staging changes involves at least two background jobs:
>>> >
>>> > - Git repository refresh
>>> > - Re-indexing repository $NAME
>>>
>>> Both jobs are triggered by the IndexChangedEvent i guess (or at least the re-indexing). This event is IMHO just too stupid, as it carries no information about /what/ changed, i just tells that /something/ in your 2.5MB index changed, thus EGit has to recalculate everything. This is an issue for me too, as you can imagine, as our index is even bigger :/
>>>
>>> >
>>> > 2. Committing changes involves at least three background jobs
>>> >
>>> > - Committing changes ( which is pretty fast )
>>> > - Re-indexing repository $NAME
>>> > - Building commit info
>>>
>>> Same here, the re-indexing is triggered by the index change. I profiled those jobs a lot, and i guess for what they do, they are pretty fast. The problem is what they do - they are forced to do much to much, as the information about the actual changes is missing.
>>>
>>> >
>>> > Both these operations take a lot of time and I would expect them to be
>>> > instant, since I'm changing 2 files out of 17.000 and egit knows which
>>> > files have changed and doesn't need to recalculate the whole
>>> > repository state.
>>>
>>> This describes what we all would like it to be :D
>>>
>>> Regards,
>>> Markus
>>>
>>> >
>>> > Am I missing out on something? I've captured yourkit snapshots of both
>>> > the 'stage' and 'commit' operations and I can provide them if needed.
>>> > Before that I'd like to know whether this is expected and I should
>>> > stop worrying and start drinking more tea or considered a performance
>>> > bug and should be entered in bugzilla .
>>> >
>>> > Thanks,
>>> >
>>> > Robert
>>> >
>>> >
>>> > [1]: 331273: Commit operations opens and reads almost all files
>>> > https://bugs.eclipse.org/bugs/show_bug.cgi?id=331273
>>> >
>>> > --
>>> > Sent from my (old) computer
>>> > _______________________________________________
>>> > egit-dev mailing list
>>> > egit-dev@xxxxxxxxxxx <mailto:egit-dev@xxxxxxxxxxx>
>>> > https://dev.eclipse.org/mailman/listinfo/egit-dev
>>> >
>>> _______________________________________________
>>> egit-dev mailing list
>>> egit-dev@xxxxxxxxxxx <mailto:egit-dev@xxxxxxxxxxx>
>>> https://dev.eclipse.org/mailman/listinfo/egit-dev
>>>
>>>
>> _______________________________________________
>> egit-dev mailing list
>> egit-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/egit-dev
>
>
>
> --
> Sent from my (old) computer
>
