Re: [mat-dev] RFC: Support the capability of analyzing multiple segmente

[Andrew Johnson <andrew_johnson@xxxxxxxxxx>]

Yi Yang,

 

This is possible but depends on how complex it is and who is providing the code, as there isn’t much development resource for MAT (more help welcome!)

 

Some considerations:

 

1. How well is the format of p1, p2, p3 documented? Is the merge just a file concatenation? It seems that even the documentation explaining the format of HPROF has been removed from Java after Java 8. https://bugs.openjdk.org/browse/JDK-8260478 MAT now handles chunked GZIP files, so that might be of interest.
2. Opening p1, p2 directly would be harder as the heap dump history view, and HPROF open etc. currently operates on only one file. How do we know how many files there are?
3. Also consider generation of these segmented files; for acquire heap dump and for MAT diagnostics?? Acquire heap dump could have a step like IBM ‘jextract’ (which is different from the recent Java command of that name) which postprocessed a core file into a zip file with extra information. The concatenation could be done for acquire heap dump. MAT diagnostics should be rare, so is not as important.
4. Android had an enhanced HPROF 1.03 format – we never directly handled it, but Android provided a hprof-conv utility, and we didn’t have many complaints, so a stand-alone utility may be acceptable.
5. As you have some contact with the JVM developers, how does HPROF handle arrays of over 4GB (or 2GB?) HPROF_HEAP_DUMP_SEGMENT has a limit of 4GB, so large Java arrays overflow this. Ideally there would be a new segment type, but a hack would be to put just one array record in the HEAP_DUMP_SEGMENT and let the segment size wrap as it overflowed. If we ever had multithreaded pass 1 parsing by segments, the parser would need to peek into a segment to see if there was an overflowing array record.
   
   

Regards,

 

Andrew Johnson

From: mat-dev <mat-dev-bounces@xxxxxxxxxxx> On Behalf Of Yi Yang via mat-dev
Sent: Wednesday, August 9, 2023 4:39 AM
To: Memory Analyzer Dev list <mat-dev@xxxxxxxxxxx>
Cc: Yi Yang
Subject: [mat-dev] RFC: Support the capability of analyzing multiple segmented heap dumps as a whole

Hi MAT community, since JDK-8306441，JVM supports two-phase segmented heap dump now:

 

- Phase 1: Generate segmented heap file such as heap.hprof.p1, heap.hprof.p2, heap.hprof.p3..., p1 contains most stuff(stack, classes,etc), p2 and all others are part of heap objects only. This is done in safepoint, application is paused.

- Phase 2: Merge segmented heap dump into a complete one, i.e. heap.hprof.p1+heap.hprof.p2+heap.hprof.p3=heap.hprof. This is done without safepoint, application is resumed now.

 

In experiments, this approach significantly reduces application pause time, but it is important to note that the total time required for the heapdump itself remains unchanged.

 

I would like to ask if MAT community is willing to support the capability of analyzing multiple segmented heap dumps (i.e., allowing users to select multiple heap files for analysis as a whole. MAT could consider performing merge and analysis or directly analyzing segmented files, which involves specific implementation details). This way, the JVM does not need to execute Phase 2, and for users, the heap dump process will be considerably faster (the JVM only needs to generate segmented files and then immediately end the heap dump)?

 

Looking forward to your feedback.

 

--

Yi Yang

 

Unless otherwise stated above:

IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU