Reviving the old thread.
I suppose that JGit has three parts at the very higher level: JGit client API, JGit server API, and JGit repository API. There's no explicit boundary, but roughly org.eclipse.jgit.api.* are client APIs, org.eclipse.jgit.transport.* is server API and some repository API, org.eclipse.lib.Repository and its dependencies are the repository API. The repository API is a low level API that the server API and the client API are built on top of. I guess JGit packages can be split like this for clarity, but it's a different problem.
For the repository API, I guess it's better to define a checked exception like JGitRepositoryException and define detailed exceptions as needed by the client API and the server API. It's a checked exception because it's the client API or the server API's responsibility to handle them for the end-users (e.g. bail out with a RuntimeException, run fsck automatically). By defining a root exception as Matthias mentioned it would become API safe when introducing a new exception. If the caller doesn't care about the details, they can catch JGitRepositoryException and just assume there's a low-level error happened.
I have never looked into the client API, but it seems it's doing a good job on handling errors from the repository API.
For the server API, I don't think it's in a good state. This part can have a good exception handling strategy and a good exception structure. For servers, at least I want to distinguish if an error is caused by the server's fault or the client's fault. Also, it's almost impossible to recover from an error in the repository API or client IO error, so it's better to use non-checked exceptions (i.e. most of the cases we just bail out and show an error message to the client). I want to have JGitServerClientException (indicates it's a client error. In the HTTP world, it's 4XX.) and JGitServerBackendException (indicates it's a server error. In the HTTP world, it's 5XX) that extend from RuntimeException.
For the exception handling strategy for the server API, I want to decouple the exception handling code from ReceivePack.java and UploadPack.java. As a Git server operator, it's important to have a server-side counters for monitoring purpose. If the exception handling code is decoupled from the protocol handlers, we can separately install the code that increments counters for errors and handle them. This also makes it possible to handle errors differently for different transports. The HTTP transport would like to produce a different HTTP status code for different errors. The SSH transport might want to write errors in stderr.
So, recap: the proposal is:
Exceptions thrown from Repository.java:
+ Exception
++ JGitRepsoitoryException
+++ ... (exceptions thrown from existing Repository.java)
Exceptions thrown from ReceivePack.java and UploadPack.java
+ RuntimeException
++ JGitServerClientException
+++ ... (exceptions that indicate the client's fault, like PackProtocolException)
++ JGitServerBackendException
+++ ... (exceptions that indicate the server's fault. JGitRepsoitoryException would be converted to this in general)
Plus, ReceivePack and UploadPack will be wrapped by an exception handling layer that catches JGitServerClientException and JGitServerBackendException.
(I'm not sure if we can do this considering the API compatibility)