Bug 473493 - Add DirectMemory Consumption
Summary: Add DirectMemory Consumption
Status: CLOSED MOVED
Alias: None
Product: MAT
Classification: Tools
Component: Core (show other bugs)
Version: 1.5   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-07-24 07:28 EDT by Benoit Lacelle CLA
Modified: 2024-05-08 14:52 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Benoit Lacelle CLA 2015-07-24 07:28:56 EDT
Hello,

DirectMemory is taking more and more importance in InMemory libraries. However, it is very difficult to tackle memory-leak issues and understand properly memory consumption in such applications as MAT analyses only the Heap.

Then, I would like to be able to associate DirectByteBuffer they direct-memory footprint in addition to their Heap.

I successfully done it by considering DirectByteBuffer instances as arrays and registering their capacity as heap footprint.

However, this breaks some unit-tests as the computed Heap size is not aligned anymore with the heap-size given pre-computed in the Heap file.

Does the MAT team consider such a feature useful?
What would be the proper way to implement it? In my implementation, which is just a POC for now, I changed the heap size associated to DirectByteBuffer instances.

Best
Comment 1 Benoit Lacelle CLA 2015-07-28 11:58:46 EDT
The issue in the tests has been fixed. I prepared a branch with the modifications (very few of them as of now). But Gerrit rejects me.

How could I push this branch? This would at least enable me working on this through my different dev env.

Thanks
Comment 2 Krum Tsvetkov CLA 2015-09-22 10:39:21 EDT
Hi Benoit,

Sorry for the delayed response. I find the proposal interesting, and would like to see what you did to achieve this.

Can you give any details about the error you get from Gerrit? I would try to help if I can.

Regards,
Krum
Comment 3 Eclipse Genie CLA 2015-12-28 07:14:45 EST
New Gerrit change created: https://git.eclipse.org/r/63307
Comment 4 Kevin Grigorenko CLA 2015-12-28 10:45:27 EST
Hi Benoit, You may be interested in the free IBM Extensions for Memory Analyzer (IEMA): http://www.ibm.com/developerworks/java/jdk/tools/iema/

IEMA includes a plugin under Open Query Browser > IBM Extensions > Java SE Runtime > DirectByteBuffers. This produces a report about DirectByteBuffers, including how many are only Phantomly Retained, the fact that Oracle SDKs less than V7 add a page size to the underlying malloc, etc.

Here is a sample report:

Alignment size of 4096 (only applies for Java < 7) bytes, and word size of 8 bytes.
6 instances of java.nio.DirectByteBuffer
6 non-viewed* DirectByteBuffers. Sum capacity (with overhead)=48 (48 B)
0 non-viewed*, phantomed*** DirectByteBuffers. Sum capacity (with overhead)=0 (0 B)
6 non-viewed*, non-phantomed** DirectByteBuffers. Sum capacity (with overhead)=48 (48 B)
Maximum non-viewed*, non-phantomed** DirectByteBuffer = 8 (8 B)

=> Sum DirectByteBuffers' (0) capacity only phantomly reachable: 0 (0 B)
=> Sum DirectByteBuffers' (6) capacity strongly, softly, or weakly reachable (i.e. not only phantomly reachable): 48 (48 B)

---------
Histogram of Incoming References to strongly referenced DBBs
6 instances incoming from java.nio.DirectLongBufferU=48 (48.0 B)

---------
* A non-viewed DBB is one that has viewedBuffer=null. A viewed DBB is a duplicate or a splice of an existing DBB, which references the same underlying buffer.
** A non-phantomed DBB is one that has more than just a single sun.misc.Cleaner incoming reference.
*** A phantomed DBB is one that has just a single sun.misc.Cleaner incoming reference.
Example OQL: SELECT k, k.capacity FROM java.nio.DirectByteBuffer k WHERE ((viewedBuffer=null)and(inbounds(k).length>1))

"If the garbage collector determines at a certain point in time that the referent of a phantom reference is phantom reachable, then at that time or at some later time it will enqueue the reference. " (http://docs.oracle.com/javase/7/docs/api/java/lang/ref/PhantomReference.html).
Comment 5 Benoit Lacelle CLA 2023-02-01 05:22:59 EST
I missed this last comment. A few years later, I'm still interested in this. Where can one download given IEMA ? I see https://www.ibm.com/docs/en/support-assistant/5.0.0?topic=tools-extensions-memory-analyzer-tech-preview, but I can not find how to download such an extention for MAT.
Comment 6 Andrew Johnson CLA 2023-02-01 06:20:12 EST
See https://www.ibm.com/support/pages/eclipse-memory-analyzer-tool-dtfj-and-ibm-extensions
This is a built with the extensions.

This shows holw to install them into an existing MAT:https://publib.boulder.ibm.com/httpserv/cookbook/Major_Tools-Eclipse_Memory_Analyzer_Tool.html#installation
Comment 7 Eclipse Webmaster CLA 2024-05-08 14:52:17 EDT
This issue has been migrated to https://github.com/eclipse-mat/org.eclipse.mat/issues/25.