Search code examples
javaniobytebuffer

JAVA NIO Bytebuffer.allocateDirect() size limit over the int


I am trying to make off-heap memory buffer. I want very big size(like 10GB) buffer. I heard that jvm heap sometimes freeze because full GC. So, I try to make buffer with java.nio.ByteBuffer.

But, I met great difficulties!

java.nio.ByteBuffer.allocateDirect(int size)

function only support integer. but I want bigger size. what can I do? what should I do? please help me stack overflow gurus.

my development environment is macbook pro, i7 2.4ghz, 16gb ddr3, 250ssd, osx 10.9, eclipse kepler x64.

I try something for solve problem:

ByteBuffer buffer = ByteBuffer.allocateDirect(1024*1024*2000);
ByteBuffer buffer1 = ByteBuffer.allocateDirect(1024*1024*2000);
ByteBuffer buffer2 = ByteBuffer.allocateDirect(1024*1024*2000);
ByteBuffer buffer3 = ByteBuffer.allocateDirect(1024*1024*2000);

but this is not works. only allocate memory 1024*1024*2000 again

please help me.


Solution

  • You can't make a single buffer that big. Period. You can make several smaller ones and select amongst them as you need to in your own code. That's all you can do.

    e.g.:

    ByteBuffer[] myBuffers = new ByteBuffer[howMany];
    for (int x = 0; x < howMany; x++) {
        myBuffers[x] = ByteBuffer.allocateDirect(prettyBig);
    }
    

    and then

    byte getByte(long index) {
        int bufx = index / howMany;
        int bx = index % howMany;
        return myBuffers[bufx].get(bx); 
    }