Reading an continuously updating list in python

Question

My aim is to read values from a list in python which is continuously being updated by another process and I have tried to make a minimal working code here. I have a global array defined in cfg.py as

def initialize():
    global infrared_frames
    global depth_frames
    infrared_frames = []
    depth_frames = []

and from main.py, one thread appends a random value in this array, while the main thread sleeps for 10 seconds and then reads the value from this array. Here is the code -

from multiprocessing import Process
import random
import time
import cfg

cfg.initialize()

def update_array():
    while True:
        cfg.infrared_frames.append(random.random())

p1 = Process(target=update_array)
p1.start()
time.sleep(10)
print('starting to read array')
print(cfg.infrared_frames)
for a in cfg.infrared_frames:
    print(a)

Unfortunately, when I try to print the array before the loop and after the time.sleep() call, the array is still empty but p1 process is running. Why am I not able to read the array? Sorry if this is an easy question, my knowledge is limited when it comes to threading in python Thanks in advance. T

Answer 1

A good approach for adding and using data in multiprocessing is by using a Queue.

A minimal working example:

import time
import multiprocessing as mp

from multiprocessing.queues import Queue


def fill_queue(queue):
    """ Endless process that fills the queue"""
    task = 0
    while True:
        time.sleep(1)
        queue.put(task)
        print(f"Added task {task}")
        task += 1


def read_queue(queue):
    """ Endless process that reads from the queue """
    while True:
        if queue.empty():
            time.sleep(0.2)
        else:
            print(f"Handling task: {queue.get()}")


if __name__ == '__main__':
    queue = Queue(maxsize=-1, ctx=mp.get_context())
    task_fill_queue = mp.Process(target=fill_queue, args=(queue,))
    task_fill_queue.start()

    read_queue(queue)

Explanation

The fill_queue function keeps adding data to the queue every second. This is then handled by the read_queue function. It will check the queue every 0.2 seconds for a new item.