Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit
N.B. Note sure what you mean with' contribute'. While the test from the previous post is good enough to demonstrate the bug in JGit (if any), I don't think it would qualify for a _real_ test to be included in any test suite.




From: R�diger Herrmann <ruediger.herrmann@xxxxxx>
To: Matthias Sohn <matthias.sohn@xxxxxxxxx>
Cc: "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 4:59 PM
Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

Thanks for looking into this. The three methods shown below are all that is necessary to run in the GarbageCollectCommandTest.

>>> BEGIN

@Test
public void testPruneOldOrphanCommit() throws Exception {
StoredConfig config = git.getRepository().getConfig();
config.setString("gc", null, "prunePackExpire", "1.second.ago");
config.setString("gc", null, "pruneExpire", "1.second.ago");
config.save();
ObjectId initial = git.getRepository().resolve("HEAD");
RevCommit orphan = git.commit().setMessage("orphan").call();
changeLastModified(orphan, subtractDays(new Date(), 365));
RefUpdate refUpdate = git.getRepository()
.updateRef("refs/heads/master");
refUpdate.setNewObjectId(initial);
refUpdate.forceUpdate();
FileUtils.delete(new File(git.getRepository().getDirectory(), "logs"),
FileUtils.RECURSIVE | FileUtils.RETRY);

git.gc().setExpire(new Date()).call();
Thread.sleep(4000);
git.gc().setExpire(new Date()).call();

assertNull(git.getRepository().resolve(orphan.name()));
}

private static Date subtractDays(Date date, int days) {
Calendar calendar = Calendar.getInstance();
calendar.setTime(date);
calendar.add(Calendar.DAY_OF_MONTH, days * (-1));
return calendar.getTime();
}

private void changeLastModified(ObjectId commitId, Date date) {
File objectsDirectory = new File(git.getRepository().getDirectory(),
"objects");
File commitObjectDirectory = new File(objectsDirectory,
commitId.name().substring(0, 2));
File commitObjectFile = new File(commitObjectDirectory,
commitId.name().substring(2));
commitObjectFile.setLastModified(date.getTime());
}

<<< END


From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: R�diger Herrmann <ruediger.herrmann@xxxxxx>
Cc: Shawn Pearce <spearce@xxxxxxxxxxx>; "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 4:32 PM
Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

can you contribute this test as a new test in GarbageCollectCommandTest ?
I'll ask Christian to have a look and investigate why this still doesn't work.

On Thu, Apr 7, 2016 at 4:21 PM, R�diger Herrmann <ruediger.herrmann@xxxxxx> wrote:
Thanks for your insights. So setting gc.prunePackExpire = 1.second.ago (or pruneExpire for JGit < 4.3) and running GC twice with some seconds delay should prune the unreferenced object, right?

I changed the code accordingly but the commit is still not garbage collected (note, I am running against HEAD)
@Test
public void testPruneOldOrphanCommit() throws Exception {
StoredConfig config = git.getRepository().getConfig();
config.setString("gc", null, "prunePackExpire", "1.second.ago");
config.setString("gc", null, "pruneExpire", "1.second.ago");
config.save();
ObjectId initial = git.getRepository().resolve("HEAD");
RevCommit orphan = git.commit().setMessage("orphan").call();
changeLastModified(orphan, subtractDays(new Date(), 365));
RefUpdate refUpdate = git.getRepository()
.updateRef("refs/heads/master");
refUpdate.setNewObjectId(initial);
refUpdate.forceUpdate();
FileUtils.delete(new File(git.getRepository().getDirectory(), "logs"),
FileUtils.RECURSIVE | FileUtils.RETRY);

git.gc().setExpire(new Date()).call();
Thread.sleep(4000);
git.gc().setExpire(new Date()).call();

assertNull(git.getRepository().resolve(orphan.name()));
}



From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: R�diger Herrmann <ruediger.herrmann@xxxxxx>
Cc: Shawn Pearce <spearce@xxxxxxxxxxx>; "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 1:26 PM

Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

in order to avoid race conditions when a pack arrives while a gc is running jgit doesn't immediately prune
garbage objects but puts them in a garbage pack which is only pruned with the next gc after a grace period
defined by gc.prunePackExpire [1] which is 1 hour by default. Before [1] this reused gc.pruneExpire
which is 2 weeks by default which led to pack file explosion, this was fixed by [1] recently.
[1] will be contained in 4.3 which I am going to release today.

[1] https://git.eclipse.org/r/#/c/69628/

-Matthias

On Thu, Apr 7, 2016 at 10:23 AM, R�diger Herrmann <ruediger.herrmann@xxxxxx> wrote:
Thanks for the hint. I wasn't aware that commits are reflogged. However, if I delete the logs directory before collecting garbage, the orphan commit is still not pruned.

Here the updated test that deletes the reflog:

@Test
public void testPruneOldOrphanCommit() throws Exception {
ObjectId initial = git.getRepository().resolve("HEAD");
RevCommit orphan = git.commit().setMessage("orphan").call();
changeLastModified(orphan, subtractDays(new Date(), 365));
RefUpdate refUpdate = git.getRepository()
.updateRef("refs/heads/master");
refUpdate.setNewObjectId(initial);
refUpdate.forceUpdate();
FileUtils.delete(new File(git.getRepository().getDirectory(), "logs"),
FileUtils.RECURSIVE | FileUtils.RETRY);

git.gc().setExpire(new Date()).call();

assertNull(git.getRepository().resolve(orphan.name()));
}



From: Matthias Sohn <matthias.sohn@xxxxxxxxx>
To: Shawn Pearce <spearce@xxxxxxxxxxx>
Cc: R�diger Herrmann <ruediger.herrmann@xxxxxx>; "jgit-dev@xxxxxxxxxxx" <jgit-dev@xxxxxxxxxxx>
Sent: Thursday, April 7, 2016 12:51 AM
Subject: Re: [jgit-dev] GarbageCollectCommand does not prune unreferenced commit

On Wed, Apr 6, 2016 at 11:19 PM, Shawn Pearce <spearce@xxxxxxxxxxx> wrote:
On Wed, Apr 6, 2016 at 2:14 PM, R�diger Herrmann
<ruediger.herrmann@xxxxxx> wrote:
> You are right of course, the shown code isn't quite what I tired. Sorry for
> the confusion. If I change it to 'unreference' the orphan commit the result
> is still the same.
>
> The correct version would reset HEAD to the previous commit so that 'orphan'
> is actually unreferenced like this:
>
> @Test
> public void testPruneOldOrphanCommit() throws Exception {
> ObjectId initial = git.getRepository().resolve("HEAD");
> RevCommit orphan = git.commit().setMessage("orphan").call();
> changeLastModified(orphan, subtractDays(new Date(), 365));
> RefUpdate refUpdate = git.getRepository()
> .updateRef("refs/heads/master");
> refUpdate.setNewObjectId(initial);
> refUpdate.forceUpdate();

Is orphan still in the reflog for master? or HEAD?

I tried your test and the orphaned commit is still referenced from both the
master's and HEAD's reflogs.

-Matthias 









_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jgit-dev


Back to the top