Community
Participate
Working Groups
Trying to open a 218GB Heap file. I get the error message Requested length of new long[2,147,483,640] exceeds limit of 2,147,483,639 I have tried checking keep unreachable objects in preferences but it fails. Any help is appreciated. Let me know what I can do to help.
I'm sorry, but you have exceeded the architectural limit of the number of objects which Memory Analyzer can handle. Memory Analyzer uses signed ints for object IDs and Java arrays indexed by an int to hold information about objects. Rewriting MAT to use longs and multidimensional arrays or other data structures would be a big job, and no one has the full time job of maintaining and enhancing MAT. https://help.eclipse.org/2020-03/topic/org.eclipse.mat.ui.help/tasks/configure_mat.html?cp=59_3_16 There has been discussion in bug 473113 here: https://www.eclipse.org/lists/mat-dev/msg00573.html https://www.eclipse.org/lists/mat-dev/msg00324.html https://www.eclipse.org/lists/mat-dev/msg00325.html https://www.eclipse.org/lists/mat-dev/msg00326.html Do you have any idea as to how many objects are actually in your heap dump, or of what types (primitive arrays, Strings, other Java objects) ? Do you have some ideas of what we could do without a full rewrite?
The heap dump is from a third party MDM product. Sorry, I don't have an idea of how many objects it has in memory. I am trying to open it up visualvm, if I succeed, I will update you with the count.
I've got an idea which is implementable but may be of limited use. What do you think of this? I can modify the HPROF parser to randomly discard a percentage of objects of certain types. If we discarded some Strings and char[] then that wouldn't break the object graph as they are leaves, but might reduce the object count enough to load the dump. I can then count those objects in the unreachable object histogram.
New Gerrit change created: https://git.eclipse.org/r/164333
Gerrit change https://git.eclipse.org/r/164333 was merged to [master]. Commit: http://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=c136df30d1b6b7fb0321092b31020f5fa9c17222
I've delivered some changes which might help, available as a snapshot build: https://www.eclipse.org/mat/snapshotBuilds.php There are now some HPROF options to discard objects in the initial parse. Try Window > Preferences > Memory Analyzer > HPROF Parser Enable discard Discard precentage: 80 Discard object type pattern: char\[\]|java\.lang\.String That should discard 80% of the Strings and char[] arrays, perhaps bringing the total number of objects to less than 2,147,483,640. The discarded objects will be counted in the Unreachable Objects histogram, together with any objects found to be unreachable in the MAT garbage collection clean process (if keep_unreachable_objects is not set). With luck the discarded objects won't stop you finding the leak, although any leak might look smaller. Other objects might still have dangling references to the discarded objects - which might cause problems for some queries. Please let me know how you get on, so we can see if this idea is useful or not, and how to make it work best.
New Gerrit change created: https://git.eclipse.org/r/164568
Gerrit change https://git.eclipse.org/r/164568 was merged to [master]. Commit: http://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=46fffae971e71b80010488f2e9ad36188ecf7988
New Gerrit change created: https://git.eclipse.org/r/164659
Gerrit change https://git.eclipse.org/r/164659 was merged to [master]. Commit: http://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=93c5169bd6fc387fd1eb2f2723d86c80eeaf7217
I've made the changes including adding some help. Please could you test a snapshot build from https://www.eclipse.org/mat/snapshotBuilds.php and see if it helps you analyze your dump. Thanks.
New Gerrit change created: https://git.eclipse.org/r/164700
Gerrit change https://git.eclipse.org/r/164700 was merged to [master]. Commit: http://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=0ce3473459ddfa825a2b7088398beaf2af517e72
I'm marking this as fixed. I hope the work-around is suitable. If you see any problems with the work-around or things are unclear in the help or preferences then please reopen this defect. If you really need more than 2,147,483,639 objects in a snapshot then please open a new bug. I think though it wouldn't get fixed unless many users hit the problem and that there was a developer with sufficient time to work on the problem.
I have instance count from jproiler, the number is 2656093430.
Sorry, I didn't receive notifications for you messages. I will test and keep you posted.
New Gerrit change created: https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170904
Gerrit change https://git.eclipse.org/r/c/mat/org.eclipse.mat/+/170904 was merged to [master]. Commit: http://git.eclipse.org/c/mat/org.eclipse.mat.git/commit/?id=ceaf6a477223439affc86399c3ed83f331bbf21e
