Java JNA wrapper and memory consumption

Question

I have a question how JVM processes with JNA wrappers show memory consumption.

For example I run Java application which uses OpenCV JNA wrapper. Java application itself consumes for example 1GB of RAM and OpenCV native libs consume 3GB of RAM. So totally, when I will find appropriate Java JVM process(via ps command) it will show 1GB of RAM or 4GB(1 + 3) of RAM?

Answer 1

Your ps output will show 1GB for RSS (the Resident Set Size -- how much memory is allocated to that process and is in RAM.) The Java process will not directly show the used Native memory; however, it will appear as a portion of the VSZ (Virtual Memory Size -- all memory that the process can access, including memory that is swapped out, memory that is allocated, but not used, and memory that is from shared libraries.)

For example, I wrote the following code:

import com.sun.jna.Memory;

public class TinyJavaBigC {
    public static void main(String[] args) {
        // Grab 1 GiB of memory
        Memory buf = new Memory(1 << 30);
        // Sleep long enough to grab ps
    }
}

Regardless of the amount of Native memory reserved using new Memory(bytes) (which effectively calls malloc) the Java application consistently used the same amount memory in RSS, and limiting the Java heap size using -Xmx did not prevent allocation of native memory beyond this limit. The 1 GiB of native memory clearly disappeared from the OS's "available" memory, however.

I placed the above code in a loop incrementing the shift left value for allocation, and ran it using -Xmx512m which should have limited JVM heap to 512 MiB. The RSS, which includes all JVM Stack and Heap Memory, remained in the ~50 MiB range. The allocated memory does show up in VSZ associated with the process. As this also includes other types of memory it's not a direct measure, and vastly exceeds the available RAM and swapfile size constraints but it does at least give some indication of increased allocation.

Native Memory          RSS          VSZ
       1 byte     42.0 MiB      9.6 GiB
      2 bytes     45.8 MiB      9.6 GiB
      4 bytes     46.0 MiB      9.6 GiB
      8 bytes     46.1 MiB      9.6 GiB
     16 bytes     46.3 MiB      9.6 GiB
     32 bytes     46.5 MiB      9.6 GiB
     64 bytes     47.0 MiB      9.6 GiB
    128 bytes     47.5 MiB      9.6 GiB
    256 bytes     47.6 MiB      9.6 GiB
    512 bytes     48.9 MiB      9.6 GiB
        1 KiB     49.1 MiB      9.6 GiB
        2 KiB     49.2 MiB      9.6 GiB
        4 KiB     49.3 MiB      9.6 GiB
        8 KiB     49.3 MiB      9.6 GiB
       16 KiB     49.8 MiB      9.6 GiB
       32 KiB     50.1 MiB      9.6 GiB
       64 KiB     50.1 MiB      9.6 GiB
      128 KiB     50.6 MiB      9.6 GiB
      256 KiB     51.4 MiB      9.6 GiB
      512 KiB     51.3 MiB      9.6 GiB
        1 MiB     51.4 MiB      9.6 GiB
        2 MiB     51.4 MiB      9.6 GiB
        4 MiB     51.4 MiB      9.6 GiB
        8 MiB     51.4 MiB      9.6 GiB
       16 MiB     51.3 MiB      9.7 GiB
       32 MiB     51.3 MiB      9.7 GiB
       64 MiB     51.7 MiB      9.8 GiB
      128 MiB     51.7 MiB      9.9 GiB
      256 MiB     51.6 MiB     10.1 GiB
      512 MiB     51.6 MiB     10.5 GiB
        1 GiB     51.7 MiB     11.3 GiB
        2 GiB     51.8 MiB     12.8 GiB
        4 GiB     51.9 MiB     15.8 GiB
        8 GiB     51.9 MiB     21.8 GiB
       16 GiB     52.0 MiB     33.8 GiB
       32 GiB     52.0 MiB     57.8 GiB
       64 GiB     52.1 MiB    105.8 GiB
      128 GiB     52.1 MiB    201.8 GiB
      256 GiB     52.5 MiB    393.8 GiB
      512 GiB     52.6 MiB    777.8 GiB
        1 TiB     52.7 MiB      1.5 TiB
        2 TiB     52.7 MiB      3.0 TiB
        4 TiB     52.8 MiB      6.0 TiB
        8 TiB     52.9 MiB     12.0 TiB
       16 TiB     53.1 MiB     24.0 TiB
       32 TiB     53.2 MiB     48.0 TiB
Exception in thread "main" java.lang.OutOfMemoryError: Cannot allocate 70368744177664 bytes