SnapshotFactory.openSnapshot
└── SnapshotFactoryImpl.parse
├── HprofIndexBuilder.fill
│ ├── HprofParserHandlerImpl.beforePass1
│ ├── Pass1Parser.read
│ │ ├── read header&version
│ │ ├── read identifierSize
│ │ ├── ....
│ │ └── read anything else about objects
│ ├── HprofParserHandlerImpl.beforePass2
│ ├── Pass2Parser.read
│ │ ├── read heap segment in parallel
│ │ └── HprofParserHandlerImpl.addObject in parallel
│ ├── HprofParserHandlerImpl.fillIn
│ ├── IntArray1NWriter.writeTo
│ ├── IntIndexCollector.writeTo
│ └── LongIndexCollector.writeTo
├── GarbageCleaner.clean
├── SnapshotImpl.calculateDominatorTree
└── SnapshotImpl.calculateMinRetainedHeapSizeForClasses
HprofParserHandlerImpl.addObject
├── HprofParserHandlerImpl.prepareHeapObject
├── HprofParserHandlerImpl.mapAddressToId
├── IntArray1NWriter.log
├── IntIndexCollector.set
└── LongIndexCollector.set
--original
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,898.853 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,868.729 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,845.501 s
--without_IndexCollector_lock_and_nocompressed
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,334.532 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,300.817 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,303.18 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,330.146 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,335.981 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,335.981 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,325.994 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,333.654 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,362.704 s
testParseDump(org.eclipse.mat.tests.parser.TestParse) Time elapsed: 1,326.175 s
Thanks,
Yi YangSend Time:2023 Jun. 8 (Thu.) 01:42
Subject:Re: [mat-dev] Is it necessary to synchronize IntIndexCollector.get/set during Pass2Parser?
This would be great to see. On my analysis the majority of time was no longer in the earlier parsing stage which was running close to speed of I/O, and has moved to later stages (such as GC). So, if you see contention in earlier stages, then, yes we could look to fix it.
In terms of optimisations I think it would be great to get your opinion on these two as well, as I think they all hit the same indexing layer.
I think there's quite some overlap here and it would be good to arrive at a decision point of the efficacy in practice of the compressed arrays, with current world Java applications. The apps we use are typically large (minimum heap size is ~5GB, many apps are 100+GB), and a longer term plan to address this along with 2bn object count would be beneficial.