Search code examples
pythonnumpymultiprocessingshared-memory

Shared memory numpy array code not working

I am new to Python and am learning multiprocessing and I am following the code in the docs for Python multiprocessing.shared_memory. But I cannot get the correct value of variable c.

program1.py

>>> import numpy as np
>>> from multiprocessing import shared_memory
>>> a = np.array([1, 1, 2, 3, 5, 8])
>>> shm = shared_memory.SharedMemory(create=True, size=a.nbytes)
>>> b = np.ndarray(a.shape, dtype=a.dtype, buffer=shm.buf)
>>> b[:] = a[:]
>>> b
array([1, 1, 2, 3, 5, 8])
>>> shm.name
'wnsm_d69eb579'
>>>

program2.py

>>> import numpy as np
>>> from multiprocessing import shared_memory
>>> existing_shm = shared_memory.SharedMemory(name='wnsm_d69eb579')
>>> c = np.ndarray((6,), dtype=np.int64, buffer=existing_shm.buf)
>>> c
array([ 4294967297, 12884901890, 34359738373,           0,           0,
                 0], dtype=int64)
>>>
Solution

  • Your mistake is in the second program where you are using the wrong dtype argument. You have:

    c = np.ndarray((6,), dtype=np.int64, buffer=existing_shm.buf)

    But this should be:

    c = np.ndarray((6,), dtype=np.int32, buffer=existing_shm.buf)

    Don't forget to use shared memory methods close and unlink properly when you are through accessing shared memory so that it can be freed up.