I am trying to analyze a 200GB heapdump on a remote machine. The heap dump was created by through visualVMs "Create Heap Dump" button. Even executing it with Xmx300GB ("$(dirname -- "$0")"/MemoryAnalyzer -consolelog -application org.eclipse.mat.api.parse "$@" -vmargs -Xmx300g -XX:-UseGCOverheadLimit) causes MAT to crash with the following error:
eclipse.buildId=unknown
java.version=11.0.3
java.vendor=Oracle Corporation
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Framework arguments: -application org.eclipse.mat.api.parse ./tmp/heapdump-1577977940574.hprof
Command-line arguments: -os linux -ws gtk -arch x86_64 -consolelog -application org.eclipse.mat.api.parse ./tmp/heapdump-1577977940574.hprof
!ENTRY org.eclipse.osgi 4 0 2020-01-06 16:25:11.593
!MESSAGE Application error
!STACK 1
java.lang.OutOfMemoryError: Requested length of new long[2,147,483,640] exceeds limit of 2,147,483,639
at org.eclipse.mat.parser.index.IndexWriter$Identifier.add(IndexWriter.java:91)
at org.eclipse.mat.hprof.HprofParserHandlerImpl.reportInstance(HprofParserHandlerImpl.java:588)
at org.eclipse.mat.hprof.Pass1Parser.readPrimitiveArrayDump(Pass1Parser.java:590)
at org.eclipse.mat.hprof.Pass1Parser.readDumpSegments(Pass1Parser.java:366)
at org.eclipse.mat.hprof.Pass1Parser.read(Pass1Parser.java:175)
at org.eclipse.mat.hprof.HprofIndexBuilder.fill(HprofIndexBuilder.java:80)
at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.parse(SnapshotFactoryImpl.java:222)
at org.eclipse.mat.parser.internal.SnapshotFactoryImpl.openSnapshot(SnapshotFactoryImpl.java:126)
at org.eclipse.mat.snapshot.SnapshotFactory.openSnapshot(SnapshotFactory.java:145)
at org.eclipse.mat.internal.apps.ParseSnapshotApp.parse(ParseSnapshotApp.java:134)
at org.eclipse.mat.internal.apps.ParseSnapshotApp.start(ParseSnapshotApp.java:106)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:388)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:243)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:656)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:592)
at org.eclipse.equinox.launcher.Main.run(Main.java:1498)
at org.eclipse.equinox.launcher.Main.main(Main.java:1471)
Could the file be corrupted, or is it simply too big to be analyzed?
You have most likely encountered a known limitation of Memory Analyzer as per this comment:
"Memory Analyzer has an architectural limit of 2^31 - 3 objects, a current limit of 2^31 - 8 = 2,147,483,640 objects, but has not been tested with that many objects. The current record is a heap dump file of 48Gbytes containing 948,000,000 objects, which was opened with Memory Analyzer running with a 58Gbyte heap."
See also https://dev.eclipse.org/mhonarc/lists//mat-dev/msg00324.html
No matter how large is the heap size Java won't allow to create an array larger than Integer.MAX_VALUE - 5 which is 2,147,483,639.
You could try downloading latest Memory Analyzer version and see if the limit was improved.