Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jgit-dev] JGit-Lfs performance
  • From: Egor Baykov <Egor.Baykov@xxxxxxxxxx>
  • Date: Thu, 1 Jul 2021 19:05:49 +0000
  • Accept-language: en-US
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=silabs.com; dmarc=pass action=none header.from=silabs.com; dkim=pass header.d=silabs.com; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1NoD5+PTs55AvDhWHRLdjkcBdqAx+CQ8vJPmBdKtcOA=; b=dPn3Tj3OOjDqH1v1MJcDqnUhv+LoJYWLcAw6+CqcEqGB0esha4C8nV47zznpGHidevOyjpZyTSCmVvrbCMx/cN74fPcGD37QFI2FbHgzAOxkNUi1zJ+wzvO5eo8yh3BV+SIIyFKGjQFMzq7Mgk/+Ff2Tf0gFk9J+BCPXhnLnsgv+k01LfzB56/xdfBvqvGC7NoYS/dWGume79MOXkG/7C0BI6wVurz3GBBcqSCSd5lUjDA0z70KQZjOfknHaNT3f14cz3ZEvV9+aqpowcH6M5u6JB/BGKB8YmJKCvEE+96AT9/fo1LTU/lXMmL/Yx/S0KUShSURAjO2ZQCMeq8+0fw==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=T5qLdh1UBXQpPTTeRli+JIv2PVxvAMa17OGOjTmzy3V4J8j8BWeIQlUyXWzH92KyEia5zbSxNAG+3ITKwpXcw9M2D4BxA7Oyr2Nbev9D0QnjLlCfTz5Ka4PmPze2GdUOjonpCaAe9Al5h7VnvEJy0Vb80uOCMDY4lwj+Fcj/Bwebeg4w4b/zX2R09LMCtm84rPbeOWrMfk/jkA3pZDhZrBu3BQglEL768ttW5kRH1qOx3xRvUVTKz9JjwxbdsVN0m/BshsZHYTnrozr01sEYXw290B21bQVFp78si9fvxDycf876BEmC9uZfoIZXJx6WyBKZ2jJs/VMOq9bes/xOaA==
  • Delivered-to: jgit-dev@xxxxxxxxxxx
  • List-archive: <https://www.eclipse.org/mailman/private/jgit-dev/>
  • List-help: <mailto:jgit-dev-request@eclipse.org?subject=help>
  • List-subscribe: <https://www.eclipse.org/mailman/listinfo/jgit-dev>, <mailto:jgit-dev-request@eclipse.org?subject=subscribe>
  • List-unsubscribe: <https://www.eclipse.org/mailman/options/jgit-dev>, <mailto:jgit-dev-request@eclipse.org?subject=unsubscribe>
  • Thread-index: AddtzQUyDry4UmwDSbG4mq7s33Y4HQ==
  • Thread-topic: JGit-Lfs performance

Hello,

 

question please on LFS (org.eclipse.jgit.lfs):

 

I’m using JGit + LFS to clone the huge LFS repo located on Bitbucket Server.

The repo size is ~3 Gb with a lot of binaries there stored as LFS files.

 

Cloning of such repo using JGit+LFS takes ~20 hours.

Cloning using Windows Git cmd (with LFS support) takes ~15 mins.

 

When I clone some non-LFS repo: the performance for Git cmd vs JGit is about the same, even for the ~1GB repo.

 

All the testing is done on Windows, Bitbucket repos.

I’m using “v5.12.0.202105051250-m2” tag to clone & build JGit repo.

 

For JGit LFS I’m using built-in LFS command:

 

       BuiltinLFS.register();

       LfsFactory

       .getInstance()

      .getInstallCommand()

       .call();

 

       Git

       .cloneRepository()

       .setURI("ssh://<>")

       .setDirectory(new File("c:\\temp\\jgit"))

       .call();

 

I will really appreciate if any ideas how I could speed up the clone(checkout) process for the LFS.

 

Few more notes please:

  • I have to use the BuiltInLFS command instead of external one due to minimize the dependencies: 
    the JGit is planned to be used in a standalone application which shall not rely on any user’s installed Git.
  • I’m using ssh connection since http does not work; Seems like a know issue:  https://bugs.eclipse.org/bugs/show_bug.cgi?id=535814
  • I also tried to clone the LFS repo using EGit (5.12.0.202106070339-r) – same ~20 hours result

 

Thanks!

 

-Egor

Back to the top