Why do kfifo need smp_wmb even with spin_lock_irqsave in kernel in version 4.9.37

Question

there are two section codes in kernel lib/kfifo.c as below:

#define kfifo_in_spinlocked(fifo, buf, n, lock) \
({ \
    unsigned long __flags; \
    unsigned int __ret; \
    spin_lock_irqsave(lock, __flags); \
    __ret = kfifo_in(fifo, buf, n); \
    spin_unlock_irqrestore(lock, __flags); \
    __ret; \
})

static void kfifo_copy_in(struct __kfifo *fifo, const void *src,
    unsigned int len, unsigned int off) {
    unsigned int size = fifo->mask + 1;
    unsigned int esize = fifo->esize;
    unsigned int l;

    off &= fifo->mask;
    if (esize != 1) {
        off *= esize;
        size *= esize;
        len *= esize;
    }
    l = min(len, size - off);

    memcpy(fifo->data + off, src, l);
    memcpy(fifo->data, src + l, len - l);
    /*
     * make sure that the data in the fifo is up to date before
     * incrementing the fifo->in index counter
     */
    smp_wmb();
}

the function kfifo_in in kfifo_in_spinlocked will eventually call kfifo_copy_in function, my question is why even with spin_lock_irqsave, smp_wmb is needed? cpu got spin lock makes other cpu stall，so it can modified variables and data address exclusivly， when it unlock, others cpu can get the newest data and data address? why smp_wmb is needed?

Answer 1

Effectively, kfifo provides lock-free access in SPSC model (Single Producer Single Consumer). And the memory barrier is for concurrent accesses from a reader and from a writer.

In case of multiple producers (writers) it requires additional synchronization between writers, so only a single writer will be active at a time.

The function kfifo_in_spinlocked provides a way for such synchronization between writers: a spinlock.

Actually, locking aspects are described at the very top of kfifo.h header:

/*
 * Note about locking: There is no locking required until only one reader
 * and one writer is using the fifo and no kfifo_reset() will be called.
 * kfifo_reset_out() can be safely used, until it will be only called
 * in the reader thread.
 * For multiple writer and one reader there is only a need to lock the writer.
 * And vice versa for only one writer and multiple reader there is only a need
 * to lock the reader.
 */