How do I analyze a large heap dump of around 35-40 GB
I had to analyze a java heap dump with a size of 35-40GB and couldn’t load it on my local machine except for a remote server with a large memory.
I found Tool for analyzing large Java heap dumps as by far the best link. But after configuring everything and executing all the command lines correctly, I can’t get any report files.
My ParseHeapDump.sh file looks like it is
#!/bin/sh
#
# This script parses a heap dump.
#
# Usage: ParseHeapDump.sh <path/to/dump.hprof> [report]*
#
# The leak report has the id org.eclipse.mat.api:suspects
# The top component report has the id org.eclipse.mat.api:top_components
#
./MemoryAnalyzer -consolelog -application org.eclipse.mat.api.parse "$@" -vmargs -Xms8g -Xmx10g -XX:-UseGCOverheadLimit
and MemoryAnalyzer.ini The file looks like
-startup
plugins/org.eclipse.equinox.launcher_1.5.0.v20180512-1130.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.700.v20180518-1200
java -Xmx8g -Xms10g -jar plugins/org.eclipse.equinox.launcher_1.5.0.v20180512-1130.jar -consoleLog -consolelog -application org.eclipse.mat.api.parse "$@"
-vmargs
-Xms8g
-Xmx10g
Please let me know if I make any mistakes in the configuration, or recommend me any other tools available on the market.
Solution
Handling large heap dumps is a challenge. Both VisualVM and Eclipse Memory Analyzer require too much memory to handle heap dumps of tens of GiB.
Business analyzers show better results (especially YourKit), although I’m not sure about their actual limitations.
To routinely handle 100+ GiB, I have a headless solution for you heaplib, which is based on the codebase of VisualVM (actually Netbeans).
Heaplib is neither graphical nor interactive. It is geared towards automated reporting.
The tool allows you to write code for heap analysis using OQL/JavaScript (or Java if you prefer), but functionality is limited by memory requirements. Processing 100 GiB can take several hours, but this is acceptable for non-interactive workflows.