Re: [mat-dev] Memory Analyzer 1.12 and Eclipse 2021-06

[Andrew Johnson <andrew_johnson@xxxxxxxxxx>]

> OK, I did the investigation on the retained set issue, and I was a
> little hasty; it was actually some code I had in development on my end
> related to memory mapping changes. My 'blocker' is safe to ignore.
> 
> On Fri, Apr 30, 2021 at 10:01 AM Jason Koch <jkoch@xxxxxxxxxxx> wrote:
> >
> > Agree w/ plan for Java 11.
> > - From my pov the GarbageCleaner improvements are isolated enough that
> > if we want to, we can ship them.
> >
> > - The memory mapping is technically viable but I think it would be
> > good to get some other opinions, think about a strategy for our users,
> > etc. This would require a shift for user configuration between heap
> > and disk IO. It might need some heuristics to automatically decide
> > which to use, or feature flags to allow user control.
> >
> > - I see one issue that (imo) is a blocker, I think this needs some
> > further investigation, as I am seeing heaps with incomplete retained
> > sets especially from threads in
> > flight:https://bugs.eclipse.org/bugs/show_bug.cgi?id=569223
> >
> > Thanks
> > Jason
Thanks for the update on Bug 569223.
I'm still looking at the garbage cleaner improvements but I don't think 
they are quite ready yet.
The object marker uses the Java Fork/Join library, which is good, but I 
don't know the worst case memory consumption. The previous marker had a 
natural limit of one int array entry for each heap object. The new one 
stores an outbound array as a work unit, which has the advantage of 
packing more work into one ForkJoin item. However if there was a heap dump 
with lots of object arrays 4 levels deep from the GC roots then it would 
recurse to that level, then queue all those outbounds arrays, which 
potentially could be much bigger than just the number of objects in the 
snapshot.

I got a bit sidetracked by Bug 573258 
https://bugs.eclipse.org/bugs/show_bug.cgi?id=573258 as I kept on finding 
other nice to have items.

I think this is a worthwhile enhancement to the component report.
There are three additional sections for:
 Zero-Length Arrays
 Array Fill Ratios
 Primitive Arrays with a Constant Value
This involved some extra improvements in adding highlighting to findings 
of wasted memory, and warning triangle tags which go up to the component 
level to make it clear which sections to focus on. Now that is working we 
can add more sections if anyone has some good ideas.
The primitive arrays with constant value is now a quantized report 
grouping objects of the same length with same repeated value.
The fill ratio queries now directly show the wasted bytes in the backing 
arrays.
The array size query now shows the array length as well as the array size
The collection/array queries now show updates for which type of object 
they are operating on.
I have suppressed messages about unrecognized collections after 20 
messages per type.
The lazy loading of array lengths (like field are lazy loaded) makes it 
quick to do a getObject() and view its type, retained size etc. which is 
useful for OQL and calculating the members of a component.
The quantize queries now also operate on MAT Bytes as well as long and 
double.
A possible problem with adding the 'Zero-Length Arrays' and 'Array Fill 
Ratios' is that every array object ID will end up in a list inside the 
query result. Sometimes it is used - for the links 'First 10 of 46 
objects' but most are not, but they use memory until the HTML has been 
built.

Do try out the updated Top Components Report.

Krum, it it Eclipse 2021-06 M2 this week, are you able to submit what we 
have so far, say on Tuesday May 4 2021, ready for M2+3?

--
Andrew Johnson



Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU