Can I create a circular buffer on Linux? (current code segfaults)

Question

Inspired by this example for Windows. In short, they create a file handle (with CreateFileMapping) then create 2 different pointers to the same memory (MapViewOfFileEx or MapViewOfFile3)

So I tried to do the same thing with shm_open, ftruncate and mmap. I used mmap a few times in the past for memory and files but I never mixed it with shm_open or used shm_open.

My code fails on the second mmap with a segfault. I tried doing a syscall directly on both mmaps and it still segfaults :( How do I do this properly? The idea is I can do memcpy(p+len-10, src, 20) and have the first 10bytes of src be at the end of the memory and last 10 written to the start (hence circular)

#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>
int main()
{
    write(2, "Start\n", 6); //prints this
    int len = 1024*1024*2;
    int fd = shm_open("example", O_RDWR | O_CREAT, 0777);
    assert(fd > 0); //ok
    int r1 = ftruncate(fd, len);
    assert(r1 == 0); //ok
    char*p = (char*)mmap(0, len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
    assert((long long)p>0); //ok
    //Segfaults on next line
    char*p2 = (char*)mmap(p+len, len, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_FIXED, fd, 0); //segfaults
    write(2, "Finish\n", 7); //doesn't print this
    return 0;
}

Answer 1

Linux usually selects address space for mappings starting from a certain point and goes lower with each reservation. So your 2nd mmap call replaces one of previous file mappings (likely libc.so), which leads to SIGSEGV with SEGV_ACCERR - invalid access permissions. You are overwriting executable section of libc.so (that is being executed right now) with non-executable data.

Use strace to check what is going on inside:

$ strace ./a.out 
...
openat(AT_FDCWD, "/dev/shm/example", O_RDWR|O_CREAT|O_NOFOLLOW|O_CLOEXEC, 0777) = 3
ftruncate(3, 2097152)                   = 0
mmap(NULL, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0x7f134c1bf000
mmap(0x7f134c3bf000, 2097152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0x7f134c3bf000
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_ACCERR, si_addr=0x7f134c4ccc37} ---
+++ killed by SIGSEGV +++

Compare addresses you are passing around with /proc/$pid/maps file and you will see what you are overwriting.

Your mistake was to assume MAP_FIXED can be used without reserving memory beforehand. To do this properly you need to:

• Reserve memory by calling mmap with len * 2 size, PROT_NONE and MAP_ANONYMOUS | MAP_PRIVATE (and without file)
• Use mmap with MAP_FIXED to overwrite portions of that mapping with the content you need

Additionally, you should prefer using memfd_create instead of shm_open on Linux to avoid shared memory files from staying around. Unlinking them with shm_unlink doesn't help if your program crashes. This also gives you a file that is private to your program instance.