Description Nail Samatov 2021-09-02 12:18:58 EDT

Created attachment 287075 [details]
The test to reproduce the bug

I couldn't get jgit to work with private repositories on GitHub if we use builtin LFS and use HTTPS to authenticate (username + token).

Steps to reproduce:
0. Native git shouldn't be installed (use only builtin LFS). Don't use ssh to authenticate (we will use https).
1. Create any empty private (it's free now) repository in GitHub. You can check "Add a README file" checkbox when create a new repository.
Copy URL for the newly created empty private repository.
3. Open attached LfsPrivateRepoTest, modify URI, USERNAME, ACCESS_TOKEN to your values.
4. Run the test.
It will fail even on push step.

If you want to re-run the test, delete repository from github and repeat steps 1-4.


I investigated, why jgit's built in LFS doesn't work and found these issues:

1. Builtin LFS doesn't authenticate uploads and downloads if we don't use SSH.
According to: https://github.com/git-lfs/git-lfs/blob/main/docs/api/authentication.md
"The Git LFS API uses HTTP Basic Authentication to authorize requests."

Native git gets credentials from git credentials helper and caches them in memory. In jgit we can reuse CredentialsProvider used in some cases.
For checkout/reset operations (or if we use low-level api directly) we need a way to provide credentials explicitly (because jgit can want to download absent LFS-tracked files).

2. If we fix mentioned above issue, we find that we still can't push to github because of pre-receive hook.
Due to my investigation, it's because of absence of "verify" action in jgit.
According to:https://github.com/git-lfs/git-lfs/blob/main/docs/api/basic-transfers.md
"The Batch API can optionally return a verify action object in addition to an upload action object. If given, The Batch API expects a POST to the href after a successful upload."
If we add verify action to jgit, push is successful.

3. If we add LFS-tracked file in both branches and use git.push().setPushAll() to push all those changes, we fail to push because only a file from the first branch is uploaded to LFS server and second file isn't uploaded, so we don't pass validation.
Adding walk.reset() in PrePushHook fixes this issue.

Attached patch fixes mentioned above issues.
But I'm not sure if it's good from architectural point of view.
I would appreciate if you will help me to improve the patch if it's needed.
If it's good enough, I can proceed with submitting it to Gerrit.