| Re: [mat-dev] Is it necessary to synchronize IntIndexCollector.get/set during Pass2Parser? |
|
Thank you for your interest in improving the performance of MAT. The parsing code was enhanced to have some parallelism, and some of the index code was enhanced to be thread safe. The index code in org.eclipse.mat.parser is an MAT API, so we can’t really make incompatible changes as an adopter of MAT might already be using it and presuming some thread safety. We should improve
the Javadoc to explain what is safe. I think your example is for code in HprofParserHandlerImpl.java such as: object2classId.set(index, classIndex); object2position.set(index, object.filePosition); Yes, the index is separate for separate threads, but IntIndexCollector then chooses a page based on the index giving an ArrayIntCompressed object in which to set the value. This internally has a
byte array but the value can be spread across multiple bytes as the class index is known to have leading zeroes so only certain bits need to be stored. This means two threads could access the same byte in the byte array. So, synchronization is needed. It might
be possible to use AtomicIntegerArray instead of a byte[] but that could be slower. Is there contention on the lock? The page size is 1000000 so if the number of objects in the heap is smaller than this then they will all end up on the same ArrayIntCompressed.
If the number is much larger then there might still be a lot of contention as the objects in the HPROF are often in address order so the threads might be processing objects with a similar object index, so even reducing the page size might not help. Is using
IntIndexCollectorUncompressed any faster (at the cost of more memory)? There may be similar problems with the object2position index. For more performance work we would like some reproducible tests – there are some performance tests in org.eclipse.mat.tests but we don’t run them. Regards, Andrew Johnson From: mat-dev <mat-dev-bounces@xxxxxxxxxxx>
On Behalf Of Yi Yang via mat-dev Hi all, when building hprof index, MAT first creates Pass2Parser and concurrently parses objects and adds them to ArrayIntCollector. Then, in the subsequent fillIn
routine, it reads objects from ArrayIntCollector in order and writes them to index file. The entire process is shown below: HprofIndexBuilder - Pass2Parser.read - Pass2Parser.readSegment - addObject to IntIndexCollector in parallel by calling IntIndexCollector.set(key, value) - fillIn - write IntIndexCollector to file by calling IntIndexCollector.get(key) It seems that there is no time overlap between IntIndexCollector.get and IntIndexCollector.set? Additionally, when calling ArrayIntCollector.set(key, value) in addObject,
the key is the object's ID and is unique. So, is it really necessary to synchronize get/set under synchronized protection? I observed that addObject has considerable performance overhead, and removing this lock protection results in a 19.3% performance improvement. Best regards Yi Yang
Unless otherwise stated above:
IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU |