Re: [mat-dev] Leak undetected by MAT

[Andrew Johnson <andrew_johnson@xxxxxxxxxx>]

mat-dev-bounces@xxxxxxxxxxx wrote on
03/07/2018 14:38:52:

> From: Amir Shalem> Hey
> I've found a leak possible in
OracleJDK/OpenJDK which is impossible 
> to find using MAT
> The bug report is https://bugs.openjdk.java.net/browse/JDK-8206208
> 
> I think the problem is that HPROF dump doesn't contain the 
> information needed to find the leak, and therefor the problem is in
> HeapDumper.c code, and not in MAT
> 
> The leak is that Class::newInstance() is leaking the caller, since
> it is doing caller checks, and cache the caller forever (no 
> WeakReference, which is the fix in my opinion)
> 

> 
> So this leak is reported and hopefully fixed before Java11 release,
> but the problem is that its invisible under MAT.
> 
> Because when I try to query using OQL and find those "faulty"
> classes, I don't see any members of newInstanceCallerCache under
it
> But when I'm with a debugger
I do see them.
> 
> I'm sending this email to make sure the problem is indeed in JDK core
code,
> As far as I understand the JDK
does not dump class instances, and 
> therefor we don't have the fields: http://hg.openjdk.java.net/jdk/
> jdk/file/9f62267e79df/src/hotspot/share/services/heapDumper.cpp#l1418
> 
> What do you think?

MAT does have the capability to show
instance fields of java.lang.Class objects, although this will seem a bit
of an odd concept to the average Java programmer.

For HPROF dumps of ordinary classes
MAT display the following as outbound links:

static reference fields
<class>    ->
reference to its type, the java.lang.Class class instance
<classloader> -> the class
loader
<super> -> its superclass
(except for java.lang.Object)

The static fields (references and
primitives) also appear in the 'statics' tab of the object inspector.
The '<class>' etc. links are
created by Memory Analyzer on heap dump parsing to maintain the object
graph.

There are some special classes which
are really just ordinary objects:

void
boolean
byte
char
short
int
long
float
double

as you can't have instances of them.
[The MAT OQL help says 'the heap dump can contain java.lang.Class instances
caused by proxy classes or classes representing primitive types such as
int.class or Integer.TYPE.' These 'proxy classes' might be special too.]
Those classes representing primitive types just have a '<class>'
link added by MAT, and do have instance fields, shown in the object inspector
'Attributes' tab, and the reference instance fields are shown in the 'list_objects'
query. Those instance fields are:

Type|Name        
         |Value
---------------------------------
ref |classValueMap    
    |null
ref |annotationType    
   |null
ref |annotationData    
   |null
ref |enumConstantDirectory |null
ref |enumConstants    
    |null
ref |genericInfo      
    |null
int |classRedefinedCount   |0
ref |reflectionData    
   |null
ref |classLoader      
    |null
ref |name        
         |void
ref |newInstanceCallerCache|null
ref |cachedConstructor    
|null
---------------------------------

If regular class objects also have
these fields then MAT needs to know about them to maintain the object graph.

The question is whether the information
is in the HPROF file or not. From heapDumper.cpp it appears not.

Examining MAT Pass1Parser.java https://git.eclipse.org/c/mat/org.eclipse.mat.git/tree/plugins/org.eclipse.mat.hprof/src/org/eclipse/mat/hprof/Pass1Parser.java
https://git.eclipse.org/c/mat/org.eclipse.mat.git/tree/plugins/org.eclipse.mat.hprof/src/org/eclipse/mat/hprof/Pass2Parser.java

shows that it skips over:
signers
protection domain
constant pool

but there doesn't seem to be space
for the class instance fields. If instance records as well as class records
with the same ID were generated in HPROF dumps for classes then MAT would
need some updates in parsing to tie them together.

Also, how should these extra fields
be handled in OQL etc? I don't think they should appear as static fields
in the class when inspecting an instance of the class. Should they have
regular style field names (so possibly be hidden by real statics) or strange
names '<newInstanceCallerCache>' which then won't be directly accessible
because of the OQL grammar?

Also in the MAT object model:
IClass extends IObject
IInstance extends IObject
IArray extends IObject
IPrimitiveArray extends IArray
IObjectArray extends IArray
IClassLoader extends IInstances

now the field methods are on IInstance,
so IClass doesn't have them.
Fortunately ClassImpl extends AbstractObjectImpl
so has getFields() as well as get getStaticFields().

As a comparison, IBM VM core dumps
processed with DTFJ and read by MAT have the following class instance fields
created by MAT:

Type|Name        
                 |Value
----------------------------------------------
long|<vmRef>    
                  |1797104640
ref |<classLoader>    
            |null
ref |<protectionDomain>  
         |null
ref |<classNameString>  
          |null
ref |<_cachedEnumConstantDirectory>|null
ref |<_cachedEnumConstants>
       |null
----------------------------------------------

as well as a lot of constant pool
refs.

Andrew Johnson






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU