Can a lock-free atomic write / consistent read operation be achieved on a 4 byte int using System V shared memory across language platforms?

I want to implement a lock free counter, a 4-byte int, in System V shared memory. The writer is a C++ program, the reader is a Python program. Working roughly like this:

C++ code updates counter in an atomic operation
Python code reads counter and has a consistent view of memory (eventual consistency is perfectly acceptable)
No locks are implemented to achieve this

Within the C++ language there are atomic get/update operations that allow for this and guarantee memory consistency, I believe the same is true in Python.

However, as I understand it, the assumptions in C++ regarding atomic operations do not necessarily apply to code written and compiled in another language and compiler.

Is there a way to achieve a consistent view of shared memory across languages that doesn't involve implementing low level locks?

Solution

Yes, using the atomics library, along with a suitable shared memory library (e.g. mmap or shared_memory).
This example assumes your atomic int is in the first 4 bytes of the shared memory segment.

from atomics import atomicview, MemoryOrder, INT
from multiprocessing import shared_memory


# connect to existing shared memory segment
shmem = SharedMemory(name="test_shmem")

# get buf corresponding to "atomic" region
buf = shmem.buf[:4]

# atomically read from buffer
with atomicview(buffer=buf, atype=INT) as a:
    value = a.load(order=MemoryOrder.ACQUIRE)

# print our value
print(value)

# del our buf object (or shmem.close() will complain)
del buf

# close our shared memory handle
shmem.close()

We can use ACQUIRE memory order here rather than the default SEQ_CST.

The atomicview can only be created and used with a with statement, so you will need to manually keep your buf around (and manage its lifetime correctly).

Note: I am the author of this library