Community
Participate
Working Groups
When CModelManager sees a binary file it can't identify, it calls IBinaryParser.getBinary for each binary parser available. ElfParser sees that it's not an ELF file and calls createBinaryArchive on it, which eventually constructs an AR object to see if the file exists. The AR object examines the file by opening it and calling readLine(); if it's not a real ar file, then it throws an exception which is caught by CModelManager. Unfortunately if you have a very large non-ELF binary file in your project directories which happens to not contain newlines, this causes massive memory allocation and I/O when we only really needed to look at the first few bytes. I have a patch that fixes this problem. It makes CModelManager call parser.isBinary before we actually do a getBinary. To get that to work correctly, I had to fix an issue in the code that reads the file header hints; we must ensure that the number of bytes returned in the hints array is no less than the number of bytes available in the file. Currently it just calls readBytes once, which may return less 'hints' bytes even if there is more data available in the file. The patch also removes isBinary calls that are now redundant.
Created attachment 24249 [details] fix patch as described
This would be a good fix for 3.0.
patch applied
Unfortunately this patch totally breaks the MachO parser. I don't get any binaries showing up at all. If I revert CModelManager back to revision 1.95 it starts working again.
Created attachment 24453 [details] fix to detect the mach-o header correctly This should fix the problem. I don't know how this could have ever worked - must be exercising this code path for the first time. :)
Created attachment 24484 [details] patch to ensure the inputstream gets closed make sure the inputstream gets closed if an exception is thrown
Patches are in, Please verify.
Part 2 of the same problem is as follows: The ElfParser has two methods to create an IBinaryFile. 1) public IBinaryFile getBinary(IPath path) throws IOException 2) public IBinaryFile getBinary(byte[] hints, IPath path) throws IOException The second method is called from CModelManager. This flow checks that the file is indeed a binary or archive by reading the hint bytes. However, the flow which calls the method 1 does not read hint bytes from the candidate file. Hence, the problem which is already solved for the flow in the method 2 again occurs for method 1. To fix this, I have contributed a patch which reads hint bytes from the candidate file in the constructor of AR. The number of bytes read is not the same as the hint number indicated by the ElfParser (i.e. 128 bytes) but just enough to cover the AR magic string "!<arch>" (i.e. 7 bytes). ------------------- Also, the patch fixes the method: private String removeBlanks(String str) from class ARHeader. The problem encountered here was unsafe decoding of int. removeblanks(...) was only removing trailing spaces. The method now uses trim() which does a better job.
Created attachment 88376 [details] Read magic bytes from the candidate archive file in constructor of AR
(In reply to comment #9) > Created an attachment (id=88376) [details] > Read magic bytes from the candidate archive file in constructor of AR Thanks for the patch. I have fixed the issue re-using parts of your patch together with some performance improvements. Please verify.
