Re: [jgit-dev] Cannot garbage collect some objects

[Christian Halstrick <christian.halstrick@xxxxxxxxx>]

Hi

regarding debugging this situation. The code is in
org.eclipse.jgit.internal.storage.file.GC. Put breakpoints in doGC(),
repack() (Collects what should go into the new packfiles and should
not find your commit), and prune() (should delete loose objects,
outdated packfiles. Should delete your commit from loose objects)


On Fri, Mar 29, 2019 at 11:25 AM Christian Halstrick
<christian.halstrick@xxxxxxxxx> wrote:
>
> Hi,
>
> you may also want to add a "command.setPackExpire(null)"
>
> lets make sure the big object is still in the repo after JGits gc.
> Maybe the repo is still big because of other reasons. So directly
> after a JGit clone you should check:
>
> - are there still loose objects? Should not be. What does 'git
> count-objects -v' output
> - there should be two pack files in .git/objects/pack if everything
> went fine. Are there more?
> - is the bad big commit/file referenced from some special refs keeping
> it from being garbage collected? Check .git/*HEAD files for containing
> your objectID and the index for not referencing your big content
> - do the reflogs contain your big commit? 'git reflog show --all |
> grep -i <First6CharsOfBadCommitID>'
> - the two new packfiles created by gc should not contain your bad
> commit. Does the command 'find .git/objects/pack/*.idx | xargs git
> verify-pack -v | grep <courCommitID>' find something?
>
> Ciao
>   Chris
>
>
> On Thu, Mar 28, 2019 at 7:07 PM Jordan Place <jplace@xxxxxxxxxxxxxx> wrote:
> >
> > I am working on a JGit server that stores bare repositories on disk. I'd like to limit the size of these repositories by rejecting pushes that bring a repository over some threshold.
> >
> > I've written a JGit PreRecevieHook that calculates the size of a bare repository on disk. If a receiving repository becomes too large, the push is rejected. This mostly works great. I've tested by checking in a very large file to a local clone and then attempting to push this file to my server. I see the push is rejected with my helpful error message.
> >
> > There is one problem. Let's say after my push is rejected, I decide to abandon the recent commit with the new, big file. In theory, I could do this by resetting HEAD to a commit without the file. To remove the file from git entirely, I could run the following commands locally:
> >
> > git reflog expire --expire-unreachable=now --all
> >
> > git gc --prune=now
> >
> >
> > I can confirm the large file is gone because the size of my repository on my local disk is now small.
> >
> > If I push again, my git server indicates my repository is still too big. I can see that the problem is that, in the bare repository on the server, the large file object is still in git. This file should be unreachable via a ref though, so I should be able to remove it by running garbage collection. Indeed, running git gc --prune=now on the server removes the blob and brings the repository back to a small size. I am then able to push again.
> >
> > I tried translating this git gc command into a JGit command:
> >
> > try (Git git = new Git(repository)) {
> >     GarbageCollectCommand command = git.gc();
> >     command.setExpire(null);
> >     try {
> >         command.call();
> >     } catch (GitAPIException e) {
> >         e.printStackTrace();
> >     }
> > }
> >
> > Unfortunately, this did not remove the large file like the CLI version did.
> >
> > I am on the latest release of JGit. Am I using GarbageCollectCommand incorrectly? Is there a good way to debug why JGit does not think this object can be garbage collected?
> >
> > Thank you for your time and expertise!
> >
> > - Jordan Place
> > _______________________________________________
> > jgit-dev mailing list
> > jgit-dev@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> > https://www.eclipse.org/mailman/listinfo/jgit-dev