Is there a way to memory-map files in Java that are larger than Integer.MAX_VALUE?

Question

FileChannel#map, which is used to map a file (i.e., initiate a memory map), takes a long as length parameter.

Yet, the documentation on FileChannel#map says the following:

size - The size of the region to be mapped; must be non-negative and no greater than Integer.MAX_VALUE

First of all, why do they use a long in the first place if they only allow values up to Integer.MAX_VALUE? Is it somehow possible to map files larger than that at once?

For example, if I would like to map a 10GB file, I would like to write something like this (which ends up in an InvalidArgumentException due to the too large length):

long length = 10_000_000_000;
MappedByteBuffer buffer = (new RandomAccessFile("./memory.map", "rw")).getChannel().map(FileChannel.MapMode.READ_WRITE, 0, length);

Do I have to create consecutive memory maps for that, e.g., 5 memory maps each mapping 2GB of the given file?

Answer 1

FileChannel#map will return an instance of ByteBuffer, (MappedByteBuffer) and if you look at it, you can see that it's methods exclusively use int to index the data, e.g:

abstract byte get(int index) Absolute get method.

Which defines the limit for map itself, too.

So, you would have to either map your file piece by piece or use some existing library, e.g. https://github.com/xerial/larray, to map a bigger file, which uses JNI to map files > 2GB, and provides it's own Buffer abstraction, using long data pointers.

One more option is to use Unsafe operations, like described here.