My spark application is running on remote machine in our internal lab. To analyse the memory consumption of remote application, attached the remote application pid to JProfiler by using the 'attach mode' (with help of jpenable) from my local machine.
After attaching the remote application to JProfiler in local machine, the JProfiler showing only 5% of memory consumption of the remote machine but when we ran the 'top' command on remote Centos machine, the 'top' command showing the 72% of memory consumption. And I am unable to find the whole 72% consumption with JProfiler application.
Please help me to get the whole memory consumption (i.e., 72% of memory usage) statistics by using the JProfiler application.
top shows memory reserved by the JVM, not the actually used heap, so you cannot compare the two values.
In addition, the JVM uses native memory that does not show up in the heap. A Java profiler cannot analyze that memory.