mmap() fails when allocating large amounts of memory

Question

For my program I need an array of bytes of the size of 1/8th of the processs virtual memory space.

I used the getrlimit() system call to get the virtual memory size, then set it to the maximum limit using setrlimit(). I then used mmap() to allocate an array the size of 1/8th of the virtual memory size. Like so:

struct rlimit mem_limit;
if(getrlimit(RLIMIT_AS, &mem_limit) != 0){
    return -errno;
}
mem_limit.rlim_cur = mem_limit.rlim_max;
if(setrlimit(RLIMIT_AS, &mem_limit) != 0){
    return -errno;
}
array_size = (mem_limit.rlim_cur)/8;
printf("memory size is %lu bytes, array size is %lu bytes\n", mem_limit.rlim_cur, array_size);
mem_array = (char*) mmap(0, array_size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);
if(mem_array == MAP_FAILED){
    printf("mmap failed with %d. allocation size = %lu\n", errno, g_shadow_mem_size);
    return -errno;
}

mmap() fails here with errno 12, which as far as I know means there's not enough memory. I don't understand why since the program barely allocates memory other than this, let alone the other 7/8th of the memory.

I tried using malloc(), specifying an offset for mmap(), using the soft limit instead of the hard limit, allocating 1/32 of the memory instead of 1/8, using MAP_NORESERVE in the flags - nothing works so far. I tried running a simple test program that only does the mmap() and no other memory allocations and it doesn't work either.

This is what I get:

memory size is 18446744073709551615 bytes, array size is 2305843009213693951 bytes mmap failed with 12. allocation size = 2305843009213693951

Answer 1

The manpage of getrlimit has the following explanation.

RLIMIT_AS
  This is the maximum size of the process's virtual memory (address space).

It doesn't return the available memory size but the memory size (18EB) in the 64-bit address space.

If you need to find out the available memory size, you can use the sysinfo function.

#include <stdio.h>
#include <sys/time.h>
#include <sys/resource.h>
#include <linux/kernel.h>
#include <sys/sysinfo.h>

int main()
{
    struct rlimit m;
    struct sysinfo s;
    getrlimit(RLIMIT_AS, &m);
    printf("getrlimit rlim_cur:%ld, rlim_max:%ld\n", m.rlim_cur, m.rlim_max);
    sysinfo(&s);
    printf("sysinfo totalram:%ld, freeram:%ld, totalswap:%ld, freeswap:%ld\n", s.totalram, s.freeram, s.totalswap, s.freeswap);
    return 0;
}