| Re: [mat-dev] Memory Analyzer 1.15.0 |
|
Anton, Thank you for your comments. Actually, a bit of that happens already. The “Leak Hunter” report calls the “Heap Dump Overview” report and the “Find Leaks” query (which is actually the command “leakhunter”
from the file LeakHunterQuery.java). You can invoke “Find Leaks” from “Leak Identification” > “Find Leaks” and if you press shift / control / command? while clicking it will come up with the arguments wizard. See
From the latest development version of MAT (as things have changed a little). Report potential memory leaks. The query analyzes the dominator tree and searches for big memory chunks (by default more than 10% of the total heap). These could be single objects or
groups of objects from the same class. Then it tries to automatically find the exact accumulation point - usually an array or a collection. Arguments: -threshold_percent Big memory chunk size in percent of the heap. (Default value is 10%.) -max_paths Number of paths to garbage collection roots. (Default value is 10000.) -skip A regular _expression_ specifying which dominators/referers to skip when a problematic object is described. -excludes Fields of certain classes which should be ignored when finding paths. For example this allows paths through Weak or Soft Reference referents to be ignored. The ‘-excludes’ argument is used for calling the hidden ‘find_leaks’ query which looks for big items in the dominator tree. The excludes is used when there are multiple top level dominator tree items
of the same class. The query then looks for a common path to the GC roots including most of them, ignoring those excludes. The ‘-skip’ argument is used when calculating the ‘The instance is referenced by’ text in the suspect. E.g.
102 instances of
java.util.zip.ZipFile$Source, loaded by <system class loader> occupy
3,297,000 (11.50%) bytes.
Biggest instances:
Most of these instances are referenced from one instance of
java.util.HashMap$Node[], loaded by <system class loader>, which occupies
4,304 (0.02%) bytes. The instance is referenced by sun.net.www.protocol.jar.URLJarFile @ 0x604814958, loaded by
<system class loader>.
Keywords
If I rerun the report and change ‘-skip’ from: Problem Suspect 2
102 instances of
java.util.zip.ZipFile$Source, loaded by
<system class loader> occupy
3,297,000 (11.50%) bytes.
Biggest instances:
Most of these instances are referenced from one instance of
java.util.HashMap$Node[], loaded by
<system class loader>, which occupies
4,304 (0.02%) bytes. The instance is referenced by
org.eclipse.osgi.storage.bundlefile.ZipBundleFile @ 0x606128cb0, loaded by
tools.mat4862649:Equinox Startup Class Loader.
Keywords
So, it looks like that ‘-skip’ option would be useful to some people, and the question is how to expose it. The suspects.xml invokes ‘leakhunter’ without any arguments.
<query
name="%suspects.leaks">
<command>leakhunter</command>
</query> Changing that to:
<command>leakhunter
-skip ${skip}</command>
</query> would allow a skip argument to be provided, but if it wasn’t provided then I think it would be set to nothing and break the normal skip processing for java. etc. Adding this a would set a default, which would override the default in the command (currently they are the same but might not be if you wrote your own report.)
<param
key="skip"
value="
java\..*|javax\..*|com\.sun\..*|jdk\..*"></param> Changing that to:
<command>leakhunter
${args}</command>
</query> would allow additional arguments to be provided. From the UI, if you press ‘shift’ while clicking on the ‘Leak Suspects’ menu item you get a query wizard. Hidden query to run a report already registered in a plugin. Arguments: extensionIdentifier The bundle name and report extension identifier to find the report. The format is like: my.bundle.name:reportid -params Additional parameters for the report, as 'key=value'. So, you could then supply ‘-skip=java\..*|javax\..*|com\.sun\..*|jdk\..*|sun\.net\..*’ It looks like we could make some improvements here, though writing some clear documentation which people will see to read and understand might be the hardest. There are a few other queries with a ‘-skip’ option, including ‘Immediate Dominators’ and ‘Big Drops in Dominator Tree’. Is the current default the most useful setting for most people or should it
exclude more (sun\.net\..* etc.?) It would be harder to have a MAT UI preference for all of them as the default values are set when the queries are first loaded, and how should that be specified in batch mode? Regards, Andrew Johnson From: mat-dev <mat-dev-bounces@xxxxxxxxxxx>
On Behalf Of Anton Matsiuk via mat-dev Hello, Kevin, Andrew, and all. As a person in our workplace team who loves to dive deeply into mem dumps made out of Tomcats when OOM pops out from nowhere, I may say a big thank you for this report. It actually makes complicated things much easier. Yes, you usually just open up the hprof and press the Leak Suspects button just to have a Bird's-eye view. I may say that it is not so often a main reason of some OOM is visible from the Leak Suspects report alone without additional instruments like Dominator Tree, Histogram, Objects views, Classes views, and, of course, Thread reports are all
very useful. You mentioned VisualVM asks for app packages/classes and suggests excluding instrumentation of default Java things. I will visualize it with this screenshot:
You can see at the bottom right radio buttons, which make instrumentation of Java things excluded or included. I used this a week or two ago to find out a slow method of our application, but as I excluded Java packages, I saw just a "big number of our method self-time is all we see." When I repeated Profiling with Java packages included, I had a different output, and it pointed out I used RandomAccessFile (standard Java class) in the slightly wrong way. After a small rewrite, my colleagues is now doing a work in 5 minutes that was taking 2 hours earlier. I want to emphasize that there is a way to make a "product-level suggestion" - provide a package name of the "application which we are making." There is also a way to suggest excluding standard Java things out of scope, but this should be optional. I think every application could be divided into 3 parts. Something that is controllable 1) some framework and 2) _our code on top of it_. And something out of our direct control 3) 3rd party libs. Actually, 1 is like 3, but 2 is directly relies on 1 and it's configuration. Our code relies on the capabilities of the framework. I believe making a suggestion to report to try hard and process _our code_ using a package name is a good way to improve the Leak Suspects report further. Anton Matsiuk
Unless otherwise stated above:
IBM United Kingdom Limited Registered in England and Wales with number 741598 Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU |
