Bug 549907 - Gerrit/JGit fail to transfer output longer than 64K, instead causing "fatal: protocol error: bad line length $RANDOM_FOUR_CHARACTERS" messages
Summary: Gerrit/JGit fail to transfer output longer than 64K, instead causing "fatal: ...
Status: NEW
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-08-08 19:48 EDT by Elijah Newren CLA
Modified: 2019-08-09 12:00 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Elijah Newren CLA 2019-08-08 19:48:01 EDT
I apologize that I do not have a jgit-only reproduction recipe for this, but when reported to the gerrit folks, Matthias Sohn said this was a JGit bug.

The original thread is over at https://groups.google.com/d/msg/repo-discuss/ojIWungRVow/wtF5jDymEwAJ.  I know, I should have reported back then, but slacked again and didn't report until we hit it again today.

The basic idea to reproduce:

Install gerrit, and create a ref-update hook that produces over 64K output in an error message.  (e.g. have it check new commits and have it report a one-line summary of the bad ones, which only fails when someone pushes up thousands of new commits all at once and most all of them are "bad")  Upon attempting to push, the user will see:
  fatal: protocol error: bad line length character: oops
where 'oops' is actually just any random four characters from the output (which four it picks is based on the length of the message).


Exact reproduction example:

Here's a simple example ref-update hook which will fail all pushes with just over 64K of output which yields the above:

   #!/usr/bin/env python
   raise SystemExit('z'*5+'oops'+'z'*(65536-5))

If this is installed as $INSTALLATION_DIR/hooks/ref-update on the gerrit-server, then the user will see e.g.

   $ git push origin HEAD:refs/for/master
   Counting objects: 3, done.
   Delta compression using up to 8 threads.
   Compressing objects: 100% (2/2), done.
   Writing objects: 100% (3/3), 308 bytes | 0 bytes/s, done.
   Total 3 (delta 1), reused 0 (delta 0)
   remote: Resolving deltas: 100% (1/1)
   remote: Processing changes: refs: 1, done
   fatal: protocol error: bad line length character: oops


I've been able to trigger this going back to gerrit-2.6 or so up to gerrit-2.15.6 (current version we're running and latest I tried).
Comment 1 Matthias Sohn CLA 2019-08-08 20:08:32 EDT
looks like this fails when the error size exceeds the max packet size of the git protocol side-band-64k capability [1]

[1] https://github.com/git/git/blob/master/Documentation/technical/protocol-capabilities.txt#L123
Comment 2 Elijah Newren CLA 2019-08-09 12:00:00 EDT
If it helps at all, back when I first discovered this we also had a gitolite server (which uses git.git rather than jgit), so I put a similar pre-receive hook in place that output more than 64K, and verified that when pushing to that gitolite repository it sent me the >64K output as an error message with no problems.