Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jgit-dev] Distributed Git Server using JGit

Hi,

I'm creating a Git server, and I'd like to use JGit as implementation. JGit
contains a module called `org.eclipse.jgit.http.server` which allows to achieve
this easily via GitServlet[1]. However, I need the Git server to be clustered,
to provide a scalable solution. I've two possible solutions, but I want
to have your opinions about them.

Solution 1: N GitServlets + 1 NFS
Use N Git servlets and share the same network filesystem. Each server
points the same file system in the network. This solution is used by GitLab,
see https://docs.gitlab.com/ee/administration/high_availability/nfs.html
Personally, I'm afraid of concurrent file access to Git repository, which leads
to data corruption. According to this post[2], Git has mechanism to protect
itself, e.g using index lock. But a Git bare repository does not have index,
right? I'm confused.

Solution 2: N GitServlets + N DfsRepository + KeyValue DB
JGit provides an abstract class `DfsRepository`[3] to create a DFS repository.
This solution is used by Palantir[4] and Google[5], where data is stored in a
distributed database. I think this solution is for big company, and requires complex
setup. I don't have confidence to be able to implement DfsRepository correctly
and maintain an extra DB.

My implementation will be used by thousands of repositories, but only a few of
them are actively used. Therefore, the concurrent access should be very limited.

I'd like to have your comment about this subject.

Thanks,
Mincong

[1]: http://download.eclipse.org/jgit/site/4.11.0.201803080745-r/org.eclipse.jgit.http.server/apidocs/org/eclipse/jgit/http/server/GitServlet.html
[2]: https://stackoverflow.com/questions/13039150/are-concurrent-operations-possible-with-git-repositories
[3]: http://download.eclipse.org/jgit/site/4.11.0.201803080745-r/org.eclipse.jgit/apidocs/org/eclipse/jgit/internal/storage/dfs/DfsRepository.html
[4]: https://medium.com/@palantir/stemma-distributed-git-server-70afbca0fc29
[5]: https://www.eclipsecon.org/2013/sites/eclipsecon.org.2013/files/Scaling%20Up%20JGit%20-%20EclipseCon%202013.pdf

Back to the top