Re: [jgit-dev] NegativeArraySizeException in PackIndexV2

[Marc Strapetz <marc.strapetz@xxxxxxxxxxx>]

There are two patches waiting to be merged:

https://git.eclipse.org/r/#/c/31501/
https://git.eclipse.org/r/#/c/31752/

-Marc


On 13.08.2014 10:42, Marc Strapetz wrote:
> I've pushed a small patch, addressing both problems:
> 
> https://git.eclipse.org/r/#/c/31501/
> 
> -Marc
> 
> On 13.08.2014 03:42, Alex Blewitt wrote:
>> See also 
>> http://stackoverflow.com/questions/3038392/do-java-arrays-have-a-maximum-size
>>
>> Which suggests that MAX_VALUE might be the wrong test here. 
>>
>> Alex 
>>
>> Sent from my iPhone 5
>>
>>> On 12 Aug 2014, at 14:34, Robin Rosenberg <robin.rosenberg.lists@xxxxxxxxxx> wrote:
>>>
>>>
>>> I missed, since there is an explicit overflow check that seems ok, I'd guess corruption now.
>>>
>>> -- robin
>>>
>>> ----- Ursprungligt meddelande -----
>>>> Från: "Marc Strapetz" <marc.strapetz@xxxxxxxxxxx>
>>>> Till: "Robin Rosenberg" <robin.rosenberg.lists@xxxxxxxxxx>
>>>> Kopia: jgit-dev@xxxxxxxxxxx
>>>> Skickat: tisdag, 12 aug 2014 15:09:27
>>>> Ämne: Re: [jgit-dev] NegativeArraySizeException in PackIndexV2
>>>>
>>>>> More than 107 million objects. That would cause this behavior. That would
>>>>> imply a idx file of nearly 4 GB (36 bytes per object). See
>>>>> https://www.kernel.org/pub/software/scm/git/docs/technical/pack-format.txt
>>>>>
>>>>> By contrast, the linux kernel repo idx file contains only 2.6 M objects and
>>>>> occupies ~73 MB on disk.
>>>>>
>>>>> Corruption is a possibility. C Git would probably choke on the file if it
>>>>> was corrupted.
>>>>> The index file can be regenerated from the pack using git index-pack.
>>>>
>>>> Thanks, Robin. For now, I'll try to distinguish between both cases and
>>>> throw different Exceptions here; once we get a new bug report, I'll come
>>>> back with additional information.
>>>>
>>>> -Marc
>>>>
>>>>
>>>>
>>>>> On 12.08.2014 14:57, Robin Rosenberg wrote:
>>>>>
>>>>>
>>>>> ----- Ursprungligt meddelande -----
>>>>>> Från: "Marc Strapetz" <marc.strapetz@xxxxxxxxxxx>
>>>>>> Till: "Robin Rosenberg" <robin.rosenberg.lists@xxxxxxxxxx>
>>>>>> Kopia: jgit-dev@xxxxxxxxxxx
>>>>>> Skickat: tisdag, 12 aug 2014 13:24:19
>>>>>> Ämne: Re: [jgit-dev] NegativeArraySizeException in PackIndexV2
>>>>>>
>>>>>>> Is this a ridiculously large pack?
>>>>>>
>>>>>> Unfortunately I don't have any more information than the pure
>>>>>> stacktrace. How large would be too large? Could the fanoutTable be
>>>>>> corrupted, so bucketCount < 0?
>>>>>
>>>>> More than 107 million objects. That would cause this behavior. That would
>>>>> imply a idx file of nearly 4 GB (36 bytes per object). See
>>>>> https://www.kernel.org/pub/software/scm/git/docs/technical/pack-format.txt
>>>>>
>>>>> By contrast, the linux kernel repo idx file contains only 2.6 M objects and
>>>>> occupies ~73 MB on disk.
>>>>>
>>>>> Corruption is a possibility. C Git would probably choke on the file if it
>>>>> was corrupted.
>>>>> The index file can be regenerated from the pack using git index-pack.
>>>>>
>>>>> -- robin
>>>>>
>>>>>> -Marc
>>>>>>
>>>>>>> On 12.08.2014 11:28, Robin Rosenberg wrote:
>>>>>>>
>>>>>>>
>>>>>>> ----- Ursprungligt meddelande -----
>>>>>>>> Från: "Marc Strapetz" <marc.strapetz@xxxxxxxxxxx>
>>>>>>>> Till: jgit-dev@xxxxxxxxxxx
>>>>>>>> Skickat: tisdag, 12 aug 2014 10:50:22
>>>>>>>> Ämne: [jgit-dev] NegativeArraySizeException in PackIndexV2
>>>>>>>>
>>>>>>>> Any ideas regarding this error? Our jgit is based on 3.3.x.
>>>>>>>>
>>>>>>>> java.lang.NegativeArraySizeException
>>>>>>>>    at
>>>>>>>> org.eclipse.jgit.internal.storage.file.PackIndexV2.<init>(PackIndexV2.java:123)
>>>>>>>>    at
>>>>>>>> org.eclipse.jgit.internal.storage.file.PackIndex.read(PackIndex.java:137)
>>>>>>>>    at
>>>>>>>>    org.eclipse.jgit.internal.storage.file.PackIndex.open(PackIndex.java:96)
>>>>>>>
>>>>>>> Is this a ridiculously large pack?
>>>>>>>
>>>>>>> -- robin
>>>>>>>
>>>>>>>>
>>>>>>>> The offending code is:
>>>>>>>>
>>>>>>>> final long nameLen = bucketCnt * Constants.OBJECT_ID_LENGTH;
>>>>>>>> if (nameLen > Integer.MAX_VALUE)
>>>>>>>>  throw new IOException(JGitText.get().indexFileIsTooLargeForJgit);
>>>>>>>>
>>>>>>>> final int intNameLen = (int) nameLen;
>>>>>>>>>>> final byte[] raw = new byte[intNameLen];
>>>>>>>>
>>>>>>>> -Marc
>>>>>>>> _______________________________________________
>>>>>>>> jgit-dev mailing list
>>>>>>>> jgit-dev@xxxxxxxxxxx
>>>>>>>> To change your delivery options, retrieve your password, or unsubscribe
>>>>>>>> from
>>>>>>>> this list, visit
>>>>>>>> https://dev.eclipse.org/mailman/listinfo/jgit-dev
>>> _______________________________________________
>>> jgit-dev mailing list
>>> jgit-dev@xxxxxxxxxxx
>>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>>> https://dev.eclipse.org/mailman/listinfo/jgit-dev
>>
> _______________________________________________
> jgit-dev mailing list
> jgit-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/jgit-dev
>