Re: [jgit-dev] UnpackedObject and CorruptObjectException

[Marc Strapetz <marc.strapetz@xxxxxxxxxxx>]

> Check that the remaining field on line 287 of UnpackedObject.java is
> initialized with the same size that `git cat-file -s` says for that
> object.  It should be, because that is the size of the object from the
> object header.

The size is OK.

> Check that the SimilarityIndex actually read that many bytes, and thus
> during close() that remaining == 0.

SimilarityIndex seems to be OK as well and "remaining" there is 0.

The size of the large object is 5342681, however the corresponding file
on disk has size 7525995. Could that be a problem?

AFAIU, there are two places resulting in buffered reads: At line 398 of
UnpackedObject, the (buffered) FileInputStream "in" is wrapped into the
InflaterInputStream and buffered again. At line 157 of SimilarityIndex
bytes from that buffered stream are read again into a buffer. Both
buffers make the position of the underlying stream "in" more or less
random when UnpackedObject.checkValidEndOfStream() is invoked. First
call to Inflator.inflate() returns 0, for the empty buffer. At line 264
random bytes are read and the second call to Inflator.inflate() returns
62 and results in CorruptObjectException.

Btw, I'm referring to commit b0bfa8044a31262975a95ab11d8f52cdd387634d;
with yesterdays commit 7ba31474a334cf8f79ecd0c85598550f415b70c5 it's not
reproducible anymore.

> Hmm, something else to look at.  Line 287 overrides the JRE's
> InflaterInputStream.  But only overrides the read(byte[], int, int)
> method.  It doesn't override the other methods, which means we might
> not be updating remaining correctly for every case.  But I was pretty
> sure this was OK because we wrap it into a BufferedInputStream and we
> almost always use the read(byte[], int, int) format within JGit.

read(byte[]) is not overridden by InflaterInputStream and read()
delegates to read(byte[], int, int), so that should be no problem.

--
Best regards,
Marc Strapetz
=============
syntevo GmbH
http://www.syntevo.com
http://blog.syntevo.com



On 04.10.2010 20:03, Shawn Pearce wrote:
> On Mon, Oct 4, 2010 at 9:19 AM, Marc Strapetz <marc.strapetz@xxxxxxxxxxx> wrote:
>> I'm receiving following Exception when invoking RenameDetector. I have
>> no problems when performing "git diff" for the same commit. How should I
>> proceed with debugging?
>>
>> Caused by: org.eclipse.jgit.errors.CorruptObjectException: Object
>> 41ab2dcc9c58bebb80672f39220e74baead07e1f is corrupt: incorrect length
>>        at
>> org.eclipse.jgit.storage.file.UnpackedObject.checkValidEndOfStream(UnpackedObject.java:250)
> 
> Check that the remaining field on line 287 of UnpackedObject.java is
> initialized with the same size that `git cat-file -s` says for that
> object.  It should be, because that is the size of the object from the
> object header.
> 
> Check that the SimilarityIndex actually read that many bytes, and thus
> during close() that remaining == 0.
> 
> If the above is all true, we shouldn't be getting extra data from
> inflate on UnpackedObject.java line 244... but we are.  Which
> indicates something is wrong.  No clue what.  C Git should also be
> doing this check so I'm surprised its not failing.
> 
> Hmm, something else to look at.  Line 287 overrides the JRE's
> InflaterInputStream.  But only overrides the read(byte[], int, int)
> method.  It doesn't override the other methods, which means we might
> not be updating remaining correctly for every case.  But I was pretty
> sure this was OK because we wrap it into a BufferedInputStream and we
> almost always use the read(byte[], int, int) format within JGit.
> 
> 
> Sorry, that's all I've got for now.  The baby is starting to make
> noise that I might need to go pay more attention to...
>