[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [jgit-dev] Refs creation slowness with Jgit
|
Hello Matthias,
Thanks for your response. Here are the results of the benchmark on the
machine.
Benchmark Mode Cnt
Score Error Units
CreateFileSnapshotBenchmark.testCreateFile avgt 25
62917.171 ± 131776.325 ns/op
CreateFileSnapshotBenchmark.testCreateFileSnapshot avgt 25
3263.106 ± 126.821 ns/op
LookupFileStoreBenchmark.testLookupFileStore avgt 25
292185770.706 ± 1830241.558 ns/op
SimpleLruCacheBenchmark.readwrite avgt 25
533.560 ± 36.500 ns/op
SimpleLruCacheBenchmark.readwrite:testCacheRead avgt 25
292.723 ± 30.698 ns/op
SimpleLruCacheBenchmark.readwrite:testCacheWrite avgt 25
774.397 ± 84.068 ns/op
On 2020-09-23 15:26, Matthias Sohn wrote:
On Wed, Sep 23, 2020 at 8:05 PM <kaushikl@xxxxxxxxxxxxxx> wrote:
CC: repo-discuss@xxxxxxxxxxxxxxxx
On 2020-09-23 09:13, kaushikl@xxxxxxxxxxxxxx wrote:
Hello,
When investigating slow Gerrit NoteDB migration times, we noticed
a
slowness with jgit creating refs. During the migration a ref is
created for each Gerrit change, for example
‘changes/01/1/meta’,
‘changes/02/2/meta’, ‘changes/03/3/meta’ and so on. We see
the
slowness when creating refs in a new path. This behaviour is
repeatable, i.e each time a ref is created in a new path, it is
slow.
We noticed it is fast on some machines, but we couldn't quite nail
down why.
I have a small sample program[1] which illustrates the slowness.
It
creates two refs 'refs/heads/test_simple' and
'refs/heads/test/foo'.
On executing the program, I see output:
test/foo: 325 ms
test_simple: 4 ms
My expectation is that 'test/foo' will also be created in order of
milliseconds and not in order of few hundred milliseconds. The
slowness seems to be stemming from the Files.getFileStore(dir)
call in
FS.FileStoreAttributes.getFileStoreAttributes(Path dir).
Can you run the LookupFileStoreBenchmark [1] on this system ?
On my Mac this yields 63us to lookup a FileStore for a new file path.
After running the Maven build the benchmarks can be run using Maven:
$ cd jgit
$ mvn clean install -DskipTests -Dmaven.javadoc.skip=true
$ java -jar org.eclipse.jgit.benchmarks/target/benchmarks.jar
or in Eclipse run the main method of the LookupFileStoreBenchmark
class.
[1]
https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/refs/heads/master/org.eclipse.jgit.benchmarks/src/org/eclipse/jgit/benchmarks/LookupFileStoreBenchmark.java
-Matthias
Specs:
Jgit version is 5.9
$ java -version # also tried with version 1.8.0_252
openjdk version "1.8.0_232"
OpenJDK Runtime Environment (build
1.8.0_232-8u232-b09-0ubuntu1~16.04.1-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)
$ uname -r
4.15.0-54-generic
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial
Filesystem type is ext4
[1]
package test;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.concurrent.TimeUnit;
import org.eclipse.jgit.api.Git;
import org.eclipse.jgit.api.errors.GitAPIException;
import org.eclipse.jgit.lib.Repository;
import org.eclipse.jgit.revwalk.RevCommit;
import org.eclipse.jgit.revwalk.RevWalk;
public class Test {
public static void main(String[] args) {
try {
String path = null;
if (args.length == 1) {
path = args[0];
} else {
System.out.println("Repo path must be specified.");
System.exit(1);
}
Path repoPath = Paths.get(path);
long startTimeInNanoSecs;
long estimatedTimeInNanoSecs;
Git git = Git.init().setDirectory(repoPath.toFile()).call();
Repository repo = git.getRepository();
RevWalk walk = new RevWalk(repo);
Files.write(repoPath.resolve("file"), Arrays.asList("line"),
StandardCharsets.UTF_8);
git.add().addFilepattern("file").call();
git.commit().setMessage("create file").setAuthor("author",
"author@xxxxxxxxx").call();
RevCommit commit =
walk.parseCommit(repo.exactRef("refs/heads/master").getObjectId());
String branch = "test/foo";
startTimeInNanoSecs = System.nanoTime();
git.branchCreate().setName(branch).setStartPoint(commit).call();
estimatedTimeInNanoSecs = System.nanoTime() -
startTimeInNanoSecs;
System.out.println(branch + ": " +
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
branch = "test_simple";
startTimeInNanoSecs = System.nanoTime();
git.branchCreate().setName(branch).setStartPoint(commit).call();
estimatedTimeInNanoSecs = System.nanoTime() -
startTimeInNanoSecs;
System.out.println(branch + ": " +
TimeUnit.NANOSECONDS.toMillis(estimatedTimeInNanoSecs) + " ms");
} catch (IllegalStateException | GitAPIException | IOException
e) {
e.printStackTrace();
}
}
}
_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jgit-dev
_______________________________________________
jgit-dev mailing list
jgit-dev@xxxxxxxxxxx
To unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jgit-dev
