Use a membarrier inside a docker container

Question

I have a program which needs to be able to handle SIGTERM. For that, I need a sig_atomic_t global flag that the sigterm handler will set. And for the main code to be able to reliably read that variable, I need to use a membarrier both in the handler and in the main code.

As I have it now, it is like this:

static  int             mb_cmd;
static  sig_atomic_t    sigterm;


static
int     mb_init(void);
static
int     sigterm_init(void);
static
void    sigterm_handler(int sig);


inline
int     membarrier(int cmd, int flags)
{
        return  syscall(__NR_membarrier, cmd, flags);
}


int     main(void)
{
        int     status;

        status  = 1;
        if (sigterm_init())
                goto err;

        do {
                // do stuff
                asm volatile ("" : : : "memory");
        } while (!sigterm);

        return  0;
err:
        fprintf(stderr, "ERROR: main(): %i\n", status);
        perrorx(NULL);

        return  status;
}


static
int     mb_init(void)
{
        static bool     done = false;
        int             cmd;
        int             status;

        if (done)
                return  0;

        status  = 1;
        cmd     = membarrier(MEMBARRIER_CMD_QUERY, 0);
        if (cmd < 0)
                goto err;

        if (cmd & MEMBARRIER_CMD_PRIVATE_EXPEDITED) {
                status  = 2;
                mb_cmd  = MEMBARRIER_CMD_PRIVATE_EXPEDITED;
                if (membarrier(MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED, 0))
                        goto err;
        } else if (cmd & MEMBARRIER_CMD_GLOBAL_EXPEDITED) {
                status  = 3;
                mb_cmd  = MEMBARRIER_CMD_GLOBAL_EXPEDITED;
                if (membarrier(MEMBARRIER_CMD_REGISTER_GLOBAL_EXPEDITED, 0))
                        goto err;
        } else {
                mb_cmd  = MEMBARRIER_CMD_GLOBAL;
        }

        status  = 4;
        if (membarrier(mb_cmd, 0))
                goto err;
        done    = true;
        return  0;
err:
        fprintf(stderr, "ERROR: mb_init(): %i\n", status);
        return  status;

}

static
int     sigterm_init(void)
{
        struct sigaction        sa = {0};
        int                     status;

        status  = 1;
        if (mb_init())
                goto err;

        sigterm = false;
        membarrier(mb_cmd, 0);

        status++;
        sigemptyset(&sa.sa_mask);
        sa.sa_handler   = &sigterm_handler;
        if (sigaction(SIGTERM, &sa, NULL))
                goto err;
        return  0;
err:
        fprintf(stderr, "ERROR: sigterm_init(): %i\n", status);
        return  status;
}

static
void    sigterm_handler(int sig)
{

        (void)sig;

        sigterm = true;
        membarrier(mb_cmd, 0);
}

When I run the program on my computer it works fine, but on docker it shows the following error (errno is 1):

ERROR: mb_init(): 1
ERROR: sigterm_init(): 1
ERROR: main(): 1
./rob:
    rob.c:184:
    main():
    E1 -    Operation not permitted

How can I use a memory barrier in a program that should run on docker?

Answer 1

And for the main code to be able to reliably read that variable, I need to use a membarrier both in the handler and in the main code.

No, just make it volatile sig_atomic_t. ISO C guarantees that will make your code work without needing to write any explicit barriers in the source. (Basically like lock-free _Atomic with mo_relaxed ordering, except ordered wrt. other volatile accesses.)

And if you did need a memory barrier, you don't need a membarrier system call, just asm("" ::: "memory") to force a store or load to happen at least once in a loop.

membarrier() might be useful if you had another thread doing weakly-ordered loads from memory, but which couldn't be optimized away (hoisted out of a loop). Then membarrier() could maybe turn that relaxed load on another core into effectively an acquire load, if you do it between two stores in a producer thread.

Since you're already using a compile-time full barrier in the reader (to stop the non-volatile load from being hoisted out of the loop), and checking an exit_now or keep_running flag has no ordering wrt. other code, you don't need that.

---

ISO C only guarantees anything for volatile sig_atomic_t, not plain sig_atomic_t (which is usually just int) in the first place. The only reason to use sig_atomic_t is if you're using it with volatile.

In practice, volatile int will even be visible to other threads, not just between a signal handler and the thread that paused to run the signal handler. (Because real C implementations run on cache-coherent hardware, and don't do hardware race detection, etc.) But at that point you're just rolling your own lock-free atomics and should instead use _Atomic int. See also https://electronics.stackexchange.com/questions/387181/mcu-programming-c-o2-optimization-breaks-while-loop/387478#387478 for atomicity between a single thread and an interrupt (or signal) handler.

And see also When to use volatile with multi threading? - basically never, use C11 _Atomic from stdatomic.h. My answer explains why it does work in practice, and exactly what happens.