Re: [egit-dev] Missing tree/blob <hash> when trying to push

[Christian Halstrick <christian.halstrick@xxxxxxxxx>]

There are some alarm bells ringing in head. I was working on a bug
report where an E/JGit push fails with "Missing object ..." exception
and where a native git push works. And I have an explanation why the
native git push is slow and fixes the problem:

Imagine in your local repo you have commit x pointing to tree y. That
commit gets ammended/rebased in a way that afterwards no local ref
points anymore to this commit. Commit x and tree y are dangling. Then
garbage collection takes place and because the tree y exists already a
little bit longer than the commit x the tree is deleted but the commit
is still kept (he is not yet old enough to get garbage collected). We
have a unreferenced commit with a missing tree. Normally not a big
problem.
But during a push the server tells the client which refs he knows
about so that the client isn't sending stuff the server knows already.
And maybe on the server side there is still a ref pointing to commit
x. On client side we will produce a pack file which should include the
commit the client want's to push but which should not contain commit x
and it's children.

Now: since https://git.eclipse.org/r/#/c/31081/ or commit c4797fe9865
we have code in jgit which when producing packfiles will not only mark
certain commits as uninteresting, but explicitly also the trees these
commits are pointing to. And jgit is not checking whether the tree is
available in the local repo. If we mark a commit as uninteresting
while the referenced tree is missing ... then we will throw an missing
object exception. We always thought if the commit is there then the
tree will also be there. That's true for referenced objects, but for
unreferenced objects we are free to garbage collect trees but don't
garbage collect the commits which are referencing them.

Why is the next native git push slow and why does he solve the
problem: Maybe that push implicitly triggers a garbage collection. In
native git garbage collection can be triggered implicitly during e.g.
a push operation. And this garbage collection may now be able to
delete the commit x. And if the commit X is not existing anymore, then
even for JGit subsequent pushes are ok. We can live with the fact that
unknown commits are marked as uninteresting. But not with the fact
that we are marking known commits with missing trees.

Bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=445744

I'll propose a fix for this behaviour in JGit that we not blindly mark
trees as uninteresting but only if they exist.






Ciao
  Chris


On Thu, Oct 9, 2014 at 10:30 PM, Sascha Vogt <sascha.vogt@xxxxxxxxx> wrote:
> Hi Matthias,
>
> On 09/10/14 22:16, Matthias Sohn wrote:
>>
>> you mean final 3.5 ?
>
> I saw this (at colleagues) with 3.4.1, 3.5 RC1 and 3.5 final. As the first
> occurrence of that was quite a while ago it might even have occured on a
> 3.3.x
>
>> how about stack traces ?
>
> Not much, beside that missing <tree/object/blob> <hash>. Will get a few logs
> tomorrow. I think (iirc) that the remote server sends that back (as the
> error message indicates as well).
>
>> can you share any of the affected repositories ?
>
> Unfortunately not yet, I'm waiting for it to occur on one of our smaller or
> unimportant repositories. The point is, the repository is good again, after
> one push with C-Git. After that you can continue to work with EGit on that
> exact repository as if nothing had happened.
>
> Any idea what might help to track that down? A tcpdump of the communication
> with the server?
>
> Ah, btw: We had that with SSH as well as HTTPS as the connection mechanism.
>
>
> Greetings
> -Sascha-
>
> _______________________________________________
> egit-dev mailing list
> egit-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/egit-dev