Search code examples
queuelock-freertos

Using a lockfree spsc queue as mpmc queue on a single-core processor


When using a lockfree spsc queue on a single-core processor with RTOS, can we push/pop objects from multiple tasks/IRQs like mpmc queue? Since on a single-core system, there is only one task/IRQ can be activated in the RTOS context. It seems that pushing/popping from multiple tasks/IRQs does not violate the spsc requirement. Thanks if someone can clarify this use case.


Solution

  • I wouldn't assume that; running on a unicore machine gives you sequential consistency, but threads can still make progress in any interleaving of program order.

    That includes interrupting between any two asm instructions, potentially in the middle of a C operation like read_idx++.
    On a CISC like x86, that can (if you don't use the result) compile to a memory-destination add dword [mem], 1 which is atomic wrt. interrupts on the same core (but not other cores). But that's not even a possibility on a RISC: only load and store instructions can access memory.

    You'd have to go through the logic of the queue you're using to see if e.g. two threads could claim the same bucket with a non-atomic increment. Which as I discussed about with non-atomic increments, is a very likely problem for a circular buffer queue.

    It's always possible for another interrupt to arrive very soon after resuming user-space execution of a thread, so even badness that would require one thread to make a tiny amount of progress between something in another thread can at least theoretically happen. But non-atomicity of increments only requires a single interrupt to come in in the middle of an increment.

    If your SPSC queue was written with C++ atomics, you'd probably use things like tmp=read_idx.load(relaxed); read_idx.store(tmp+1, release); specifically to avoid an expensive atomic RMW, so then even on x86 it wouldn't compile to a memory-destination inc even if you didn't use the result.

    (Fun fact: on a unicore x86, xadd [mem], eax without a lock prefix would be useful for fetch_add where you use the return value: atomic wrt. interrupts, but not a slow atomic RMW. There's no standard way to get a C compiler to emit that, though: it's either non-atomic plain variables or _Atomic which uses lock prefixes on RMWs. You could use gcc -Wa,-momit-lock-prefix=yes to tell the assembler to omit all lock prefixes when compiling for unicore, though. The resulting code would only be usable with green-threads or in a single-threaded VM, or a single-threaded program using atomics with signal handlers. Or an x86 microcontroller or actual retro CPU.)