Re: [egit-dev] Missing tree/blob <hash> when trying to push

[Duft Markus <Markus.Duft@xxxxxxxxxxxxxxxx>]

Hey,

Ah, yes, from a technical perspective this makes sense. Thanks :)

>From a user's perspective it still feels broken, but that's none of JGit's business then :D

Cheers,
Markus

> -----Ursprüngliche Nachricht-----
> Von: Christian Halstrick [mailto:christian.halstrick@xxxxxxxxx]
> Gesendet: Freitag, 10. Oktober 2014 09:02
> An: Duft Markus
> Cc: Sascha Vogt; EGit developer discussion
> Betreff: Re: [egit-dev] Missing tree/blob <hash> when trying to push
>
> Markus: if you look at the docs of "git gc" [1]  you see that there is
> a option "prune". It means that git first finds out the set of objects
> which are unreferenced and can be deleted. But git will not delete
> those loose objects which are too young (by default 2 weeks). So, if
> you have a commit x pointing to a tree y stored in loose objects then
> it is very likely that files in the .git/objects directory are not of
> the same age. The tree is created during "git add" and the commit
> during "git commit". If you later amend the commit and e.g. only
> reword the commit message then the commit  will be modified while the
> tree stays untouched. If now a gc takes place the tree may be older
> than two weeks (-> deleted) while the commit is too young.
> I don't know all reasons why this functionality is like that. But one
> good thing with this feature is that it solves a few problems with
> race conditions and parallel execution of gc and other git commands.
> Ciao
>   Chris
>
>
> On Fri, Oct 10, 2014 at 8:18 AM, Duft Markus
> <Markus.Duft@xxxxxxxxxxxxxxxx> wrote:
> > Hey,
> >
> > Not that I'm a GIT internals pro, but from a user's perspective it sounds much like a bug that a commit without tree can
> exist. IMHO, either the GC should not be allowed to collect the tree if it is still referenced, or hast to remove the commit(s)
> also. What would a user gain from a commit without tree? He cannot possibly get much information out of it anyway... I
> mean, usually you're not only interested in metadata if you're searching for a now-unreferenced commit that you "lost"
> somehow ;)
> >
> > Cheers,
> > Markus
> >
> >> -----Ursprüngliche Nachricht-----
> >> Von: egit-dev-bounces@xxxxxxxxxxx [mailto:egit-dev-bounces@xxxxxxxxxxx] Im Auftrag von Christian Halstrick
> >> Gesendet: Donnerstag, 9. Oktober 2014 23:28
> >> An: Sascha Vogt
> >> Cc: EGit developer discussion
> >> Betreff: Re: [egit-dev] Missing tree/blob <hash> when trying to push
> >>
> >> There are some alarm bells ringing in head. I was working on a bug
> >> report where an E/JGit push fails with "Missing object ..." exception
> >> and where a native git push works. And I have an explanation why the
> >> native git push is slow and fixes the problem:
> >>
> >> Imagine in your local repo you have commit x pointing to tree y. That
> >> commit gets ammended/rebased in a way that afterwards no local ref
> >> points anymore to this commit. Commit x and tree y are dangling. Then
> >> garbage collection takes place and because the tree y exists already a
> >> little bit longer than the commit x the tree is deleted but the commit
> >> is still kept (he is not yet old enough to get garbage collected). We
> >> have a unreferenced commit with a missing tree. Normally not a big
> >> problem.
> >> But during a push the server tells the client which refs he knows
> >> about so that the client isn't sending stuff the server knows already.
> >> And maybe on the server side there is still a ref pointing to commit
> >> x. On client side we will produce a pack file which should include the
> >> commit the client want's to push but which should not contain commit x
> >> and it's children.
> >>
> >> Now: since https://git.eclipse.org/r/#/c/31081/ or commit c4797fe9865
> >> we have code in jgit which when producing packfiles will not only mark
> >> certain commits as uninteresting, but explicitly also the trees these
> >> commits are pointing to. And jgit is not checking whether the tree is
> >> available in the local repo. If we mark a commit as uninteresting
> >> while the referenced tree is missing ... then we will throw an missing
> >> object exception. We always thought if the commit is there then the
> >> tree will also be there. That's true for referenced objects, but for
> >> unreferenced objects we are free to garbage collect trees but don't
> >> garbage collect the commits which are referencing them.
> >>
> >> Why is the next native git push slow and why does he solve the
> >> problem: Maybe that push implicitly triggers a garbage collection. In
> >> native git garbage collection can be triggered implicitly during e.g.
> >> a push operation. And this garbage collection may now be able to
> >> delete the commit x. And if the commit X is not existing anymore, then
> >> even for JGit subsequent pushes are ok. We can live with the fact that
> >> unknown commits are marked as uninteresting. But not with the fact
> >> that we are marking known commits with missing trees.
> >>
> >> Bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=445744
> >>
> >> I'll propose a fix for this behaviour in JGit that we not blindly mark
> >> trees as uninteresting but only if they exist.
> >>
> >>
> >>
> >>
> >>
> >>
> >> Ciao
> >>   Chris
> >>
> >>
> >> On Thu, Oct 9, 2014 at 10:30 PM, Sascha Vogt <sascha.vogt@xxxxxxxxx> wrote:
> >> > Hi Matthias,
> >> >
> >> > On 09/10/14 22:16, Matthias Sohn wrote:
> >> >>
> >> >> you mean final 3.5 ?
> >> >
> >> > I saw this (at colleagues) with 3.4.1, 3.5 RC1 and 3.5 final. As the first
> >> > occurrence of that was quite a while ago it might even have occured on a
> >> > 3.3.x
> >> >
> >> >> how about stack traces ?
> >> >
> >> > Not much, beside that missing <tree/object/blob> <hash>. Will get a few logs
> >> > tomorrow. I think (iirc) that the remote server sends that back (as the
> >> > error message indicates as well).
> >> >
> >> >> can you share any of the affected repositories ?
> >> >
> >> > Unfortunately not yet, I'm waiting for it to occur on one of our smaller or
> >> > unimportant repositories. The point is, the repository is good again, after
> >> > one push with C-Git. After that you can continue to work with EGit on that
> >> > exact repository as if nothing had happened.
> >> >
> >> > Any idea what might help to track that down? A tcpdump of the communication
> >> > with the server?
> >> >
> >> > Ah, btw: We had that with SSH as well as HTTPS as the connection mechanism.
> >> >
> >> >
> >> > Greetings
> >> > -Sascha-
> >> >
> >> > _______________________________________________
> >> > egit-dev mailing list
> >> > egit-dev@xxxxxxxxxxx
> >> > To change your delivery options, retrieve your password, or unsubscribe from
> >> > this list, visit
> >> > https://dev.eclipse.org/mailman/listinfo/egit-dev
> >> _______________________________________________
> >> egit-dev mailing list
> >> egit-dev@xxxxxxxxxxx
> >> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> >> https://dev.eclipse.org/mailman/listinfo/egit-dev
> > Salomon Automation GmbH | Friesachstrasse 15 | 8114 Friesach bei Graz | Austria
> > Registered Office: Friesach bei Graz | Commercial Register: 49324 K | VAT no. ATU28654300
> > Commercial Court: Landesgericht für Zivilrechtssachen Graz
Salomon Automation GmbH | Friesachstrasse 15 | 8114 Friesach bei Graz | Austria
Registered Office: Friesach bei Graz | Commercial Register: 49324 K | VAT no. ATU28654300
Commercial Court: Landesgericht für Zivilrechtssachen Graz