Hello,
We found a similar bug.
But now it relates to the GC.deleteOrphans() method.
This method also unable to remove orphaned files on
Windows if the files have R/O attribute set. The
produced error is the next:
===============================================================================
2020-10-20 01:39:21,294-0400 ERROR
[o.e.j.i.storage.file.GC]
E:\xxx\objects\pack\pack-d849246de612f3d29812703e482773ee5b1453c6.idx
java.nio.file.AccessDeniedException:
E:\xxx\objects\pack\pack-d849246de612f3d29812703e482773ee5b1453c6.idx
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
at
sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at
org.eclipse.jgit.internal.storage.file.GC.deleteOrphans(GC.java:981)
at
org.eclipse.jgit.internal.storage.file.GC.repack(GC.java:882)
at
org.eclipse.jgit.internal.storage.file.GC.doGc(GC.java:267)
at
org.eclipse.jgit.internal.storage.file.GC.gc(GC.java:218)
[...]
===============================================================================
Thus if for any reasons some orphaned files remain in
the 'pack' folder -- they will remain here infinitely
and will flood the logs with the above error messages.
On 10.06.2020 3:56, Denis Malyshkin wrote:
Hello Matthias,
Thank you a lot.
I've tested with both JGit '5.8.0-SNAPSHOT'
and '5.8.0.202005061305-m2' – and I confirm
that the bug is not reproduced anymore.
On 09.06.2020 19:57, Matthias Sohn wrote:
Hello,
The simple example of code, mentioned in
the previous email.
class
Main {
private static File REPO_DIR = new
File("testRepo");
private static final String REPO_URL =
"https://github.com/denis-bigbrassband-com/testing.git";
public static final String
REF_SPEC_BARE =
"+refs/heads/*:refs/heads/*";
public static final String TAG_SPEC =
"+refs/tags/*:refs/tags/*";
public static void main(String[] args)
{
System.out.println("Start
cloning...");
CloneCommand cloneCommand = new
CloneCommand();
cloneCommand.setBare(true);
cloneCommand.setCloneAllBranches(true);
cloneCommand.setCloneSubmodules(true);
cloneCommand.setURI(REPO_URL);
cloneCommand.setDirectory(REPO_DIR);
cloneCommand.setTimeout(60);
try (Git git =
cloneCommand.call()) {
Repository repo =
git.getRepository();
System.out.println("Cloned.
Press Enter key when the repository is
changed...");
System.in.read();
System.out.println("Fetching...");
try (Transport tn =
Transport.open(repo, REPO_URL)) {
tn.setTimeout(60);
ProgressMonitor monitor =
new TextProgressMonitor();
tn.fetch(monitor,
Arrays.asList(new RefSpec(REF_SPEC_BARE),
new RefSpec(TAG_SPEC)));
}
System.out.println("Fetched.
Starting GC...");
GC gc = new
GC((FileRepository) repo);
PackConfig pConfig = new
PackConfig(repo);
pConfig.setBuildBitmaps(false); //
disable bitmap indexes
gc.setPackConfig(pConfig);
gc.setPackExpireAgeMillis(0);
gc.gc();
System.out.println("GC
completed!");
} catch (Exception e) {
e.printStackTrace();
}
}
}
Hello,
We found an issue with the JGit GC on
Windows.
I've attached a simple example to expose
this issue. Steps to reproduce:
- Clone some repository using JGit.
- Add commits to the repository on
the git server side (for example,
push commits from another computer).
- Fetch the cloned repository using
JGit.
- Call the JGit GC on the cloned
repository.
- After the GC the cloned
repository 'objects/pack' folder
contains all packs -- both existed
before the GC and created during the
GC.
- If you repeat steps 2-4 then new
pack files appears on every
iteration. And all old pack files
remain in the repository
'objects/pack' folder.
It seems this is a bug.
The logic of the bug is the following:
- The JGit clone & fetch
operations mark the created pack
files as 'read-only' in the ObjectDirectoryPackParser.parse
method:
- JGit GC operation also marks the
created pack files as 'read-only' in
the GC.writePack method:
- JGit uses Files.delete()
method in the GC.removeOldPack
to remove old pack files:
- This Files.delete()
method unable to remove 'read-only'
files on Windows (please see https://stackoverflow.com/questions/12139482/difference-between-filesdeletepath-and-filedelete),
throws an exception, but this
exception is silently ignored:
- As a result all pack files remain
in the repository, but nobody knows
that something goes wrong:
It seems that the R/O attribute
should be cleared before deleting old
pack files.
Use the nightly build 5.8.0-SNAPSHOT until
5.8 is released next Wednesday.
-Matthias