There is a common understanding that a single snapshot of the Java heap is not enough for finding a memory leak. The usual approach is to search for a monotonous increase of the number of objects of some class by “online” profiling/monitoring or by comparing a series of snapshots made over time. However, such a “live” monitoring is not always possible, and is especially difficult to be performed in productive systems because of the performance costs of using a profiler, and because of the fact that some leaks show themselves only rarely, when certain conditions have appeared.

In this blog will try to show that analysis based on a single heap dump can also be an extremely powerful means of finding memory leaks. I will give some tips how to obtain data suitable for the analysis. I will then describe how to use the automated analysis features of the Memory Analyzer tool, which was contributed several months ago to Eclipse. Automating the analysis greatly reduces the complexity of finding memory problems, and enables even non-experts to handle memory-related issues. All you need to do is provide a good heap dump, and click once to trigger the analysis. The Memory Analyzer will create for you a report with the leak suspects. What this report contains, and how the reported leak suspects are found is described below.
(more…)

Markus Kohler blogged about Analyzing Memory Consumption of Eclipse:

During my talk on May 7 at the Java User Group Karlsruhe about the Eclipse Memory Analyzer I used the latest build of Eclipse 3.4 to show live, that there’s room for improvement regarding the memory consumption of Eclipse.

He goes on to have a closer look at the spell checker and looks at duplicate strings.

Now you may think, that this guy is bashing Eclipse, but that’s really not the case.
If you beg enough, I might also take a closer look at Netbeans :]

If nothing else, it shows how relative simple it is to gain some insights about the memory of your application…

The Hotspot Virtual Machine employs generational garbage collection: a young generation holds recently created objects, the tenured generation those objects which survived (multiple) major Garbage Collections (GCs). The heap dump contains all objects from both spaces, even though we cannot tell anymore to which generation they belonged.

The permanent generation (or: perm space) is a different beast: It is used to store class and method data as well as interned strings. Just like heap space, you can also run out of perm space. That’s what happens for example if you install too many plug-ins to your Eclipse IDE: OutOfMemoryError: PermGen full. In this case, you have to increase the available perm space using -XX:MaxPermSize.

Too many plug-ins are only one reason: One can run out of perm space due to too many interned Strings or because of leaking class loaders. Increasing perm space will only help for so long.

Because a heap dump does not really contain a lot of information about perm space, perm problems are difficult to tackle. Recently, I found this great article by Sporar, Sundararajan and Kieviet. The authors shed some light on the permanent generation. Of course, I had to check right away if and how I can use the Eclipse Memory Analyzer to analyze this “unknown” generation. This is what this blog is about.
(more…)

Hello,

I just wanted to take the opportunity to thank everybody who joined our technical session at Java One 2008 - “Automated Heap Dump Analysis for Developers, Testers, and Support Employees”.
I was very happy to see more than 400 people in the room, despite of the fact that the session was in the afternoon of the very last day of the conference. Thank you! It was a real pleasure for us the speakers.

As a project that is relatively new at Eclipse (was recently contributed by SAP) we are very interested to get some feedback from you - both about our JavaOne session and about the tool.

So I hope that we can get some comments added to this blog from the people who saw the session @ Java One and want to spend several minutes to give us their valuable opinion.

And for any feedback, suggestions or comments on the tool itself, please use our newsgroups.

For anybody who is interested in the topic and in the Memory Analyzer tool, but didn’t have the chance to play with it yet - our project page is http://www.eclipse.org/mat/.

The goal of this article is not to provide you with a step-by-step approach in finding memory problems, but to reveal several powerful features of the Memory Analyzer, which are irreplaceable in memory leak hunting. These features might be unified into one topic – aggregation.

Memory Analyzer - is a powerful Java heap analyzer. It makes it possible to find a memory leak or reduce memory consumption by looking into the heap dump only. Our wiki page provides detailed information on how to acquire a heap dump.

Sometimes you open a heap dump in Memory Analyzer and see nothing … but a pile of small objects:

As overview pane shows, the majority of retained memory is more or less equally distributed among several objects, which makes it difficult to see a suspect for a memory leak at first sight.

Grouping in Dominator Tree

The pie in the overview pane is a graphical representation of the Dominator Tree, which you can open from the toolbar using the button

The Dominator Tree is built out of an object graph. Each object in the tree is responsible for keeping its children alive. The tree is sorted in descending order according to the size an object retains in memory.
Now, if we group the dominator tree by class (using the drop-down toolbar menu), we will see a different picture:

We can see that 700 objects of class com.foo.bar.menu.MenuEntry retain 46 % of the total heap space. Suddenly we see an unequal distribution of the retained memory.
How is it possible that we see it only now? The answer is that in this heap dump we do not have one single big object, but a group of leaking objects. Remember, the overview page of the heap dump only displays the distribution of retained memory heap on a per-object basis. So, when we group the dominator tree by class, we group our leaking objects to their origin class and thus get an accumulated retained heap.
Let’s find out who is referencing all those com.foo.bar.menu.MenuEntry objects. For this select List objects –> with incoming references from the context menu. As a result we will get an object reference graph:

If we drill down into the com.foo.bar.menu.MenuEntry, we can see the chain of objects referencing our suspect. Unfortunately, in this case it is not really helpful as we see only a LinkedList structure.

Grouping referrers to the class level

As we have seen looking at the incoming references graph for our suspect objects did not reveal a lot. Grouping referrers to class level might be more helpful. This operation is performed via a context menu. Back in the dominator tree grouped by class, right-click on our suspect com.foo.bar.menu.MenuEntry and select Show objects by class –> by incoming references:

Grouping referrers by class allows us to see a simple and easy to understand reference chain:

We can now see, that class com.foo.bar.menu.MenuEntry is referenced by a LinkedListEntry, which we already knew. But as we continue we see that this entry is again referenced by a LinkedListEntry (this is the head of the LinkedList) and by the LinkedList. LinkedList itself is referenced by two classes, one of which is of no interest as it is from the java.* package (which for now we assume to be unproblematic). The other one, however, is of interest.
The easiest way to find out what keeps our memory leaking group of objects alive in the heap is to check the path to the garbage collection roots. However, paths to garbage collection roots can only be found for a single object but we are interested in finding a common chain of references to a group of objects. Well, this is also possible.

Merging Paths to Garbage Collection Roots

The Merged Paths to GC Roots view shows the shortest paths from the GC roots to each instance of the selected class. You can open this heap editor using the context menu Merge Shortest Paths to GC Roots on the com.foo.bar.menu.MenuEntry class instance:

This operation explores all the paths from the different objects to their GC roots at once and finds a common chain of references for this group of objects. It is a common procedure to exclude weak and soft references from the result as they cannot be the main reason of keeping the objects of interest alive.

There we got the reason: one instance of the class com.foo.bar.menu.FoodSupplies keeps in memory 700 instances of the class com.foo.bar.menu.MenuEntry. Now is the right time to check what is going on in the code and fix the problem.

Conclusion

Apparently, Memory Analyzer has more aggregation features to offer. Some of them will be covered in our future blogs.