Re: [egit-dev] Missing tree/blob <hash> when trying to push

[Sascha Vogt <sascha.vogt@xxxxxxxxx>]

Hi Christian,

thanks for the detailed explaination and it looks like this is the
direction this "issues" comes from.

A few additional notes:
If such an issue occurs we ran "git fsck --full" and you're right, there
are usually "dangling commits" found.

We also usually manually run a "git gc" which on all but one occasion
didn't fix the issue. The next time it arises I'll test if a
    git reflog expire --expire=now --all
    git gc --aggressive --prune=now
helps.

Is there anything else I could do / test if it fixes the issue (apart
from a C-Git push)?

Maybe the workflow some colleagues are using give you a few additional
pointers.

We have a central server. Devs push to this server on a topic branch. A
second dev then pulls this topic branch, does some reviewing and after
that does a merge to master with the merge option "squash". So the
multiple topic branch commits occur as a single commit on master.

If that workflow is more likely to cause the issue, that would explain
why only a few colleagues have an issue at all. Some projects use Gerrit
and others only push to their topic branches and don't execute the
merging and squashing...

Greetings
-Sascha-

PS: I think everyone posting to the EGit mailinglist must be subscribed.
Wouldn't it be better to just use the list as a recipient instead of
huge to and cc lists? Or do you prefer to be adressed directly?

Am 09.10.2014 um 23:27 schrieb Christian Halstrick:
> There are some alarm bells ringing in head. I was working on a bug
> report where an E/JGit push fails with "Missing object ..." exception
> and where a native git push works. And I have an explanation why the
> native git push is slow and fixes the problem:
> 
> Imagine in your local repo you have commit x pointing to tree y. That
> commit gets ammended/rebased in a way that afterwards no local ref
> points anymore to this commit. Commit x and tree y are dangling. Then
> garbage collection takes place and because the tree y exists already a
> little bit longer than the commit x the tree is deleted but the commit
> is still kept (he is not yet old enough to get garbage collected). We
> have a unreferenced commit with a missing tree. Normally not a big
> problem.
> But during a push the server tells the client which refs he knows
> about so that the client isn't sending stuff the server knows already.
> And maybe on the server side there is still a ref pointing to commit
> x. On client side we will produce a pack file which should include the
> commit the client want's to push but which should not contain commit x
> and it's children.
> 
> Now: since https://git.eclipse.org/r/#/c/31081/ or commit c4797fe9865
> we have code in jgit which when producing packfiles will not only mark
> certain commits as uninteresting, but explicitly also the trees these
> commits are pointing to. And jgit is not checking whether the tree is
> available in the local repo. If we mark a commit as uninteresting
> while the referenced tree is missing ... then we will throw an missing
> object exception. We always thought if the commit is there then the
> tree will also be there. That's true for referenced objects, but for
> unreferenced objects we are free to garbage collect trees but don't
> garbage collect the commits which are referencing them.
> 
> Why is the next native git push slow and why does he solve the
> problem: Maybe that push implicitly triggers a garbage collection. In
> native git garbage collection can be triggered implicitly during e.g.
> a push operation. And this garbage collection may now be able to
> delete the commit x. And if the commit X is not existing anymore, then
> even for JGit subsequent pushes are ok. We can live with the fact that
> unknown commits are marked as uninteresting. But not with the fact
> that we are marking known commits with missing trees.
> 
> Bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=445744
> 
> I'll propose a fix for this behaviour in JGit that we not blindly mark
> trees as uninteresting but only if they exist.
> 
> Ciao
>   Chris
> 
> 
> On Thu, Oct 9, 2014 at 10:30 PM, Sascha Vogt <sascha.vogt@xxxxxxxxx> wrote:
>> Hi Matthias,
>>
>> On 09/10/14 22:16, Matthias Sohn wrote:
>>>
>>> you mean final 3.5 ?
>>
>> I saw this (at colleagues) with 3.4.1, 3.5 RC1 and 3.5 final. As the first
>> occurrence of that was quite a while ago it might even have occured on a
>> 3.3.x
>>
>>> how about stack traces ?
>>
>> Not much, beside that missing <tree/object/blob> <hash>. Will get a few logs
>> tomorrow. I think (iirc) that the remote server sends that back (as the
>> error message indicates as well).
>>
>>> can you share any of the affected repositories ?
>>
>> Unfortunately not yet, I'm waiting for it to occur on one of our smaller or
>> unimportant repositories. The point is, the repository is good again, after
>> one push with C-Git. After that you can continue to work with EGit on that
>> exact repository as if nothing had happened.
>>
>> Any idea what might help to track that down? A tcpdump of the communication
>> with the server?
>>
>> Ah, btw: We had that with SSH as well as HTTPS as the connection mechanism.
>>
>>
>> Greetings
>> -Sascha-