multiprocessing - child process constantly sending back results and keeps running

Question

Is it possible to have a few child processes running some calculations, then send the result to main process (e.g. update PyQt ui), but the processes are still running, after a while they send back data and update ui again? With multiprocessing.queue, it seems like the data can only be sent back after process is terminated. So I wonder whether this case is possible or not.

Answer 1

I don't know what you mean by "With multiprocessing.queue, it seems like the data can only be sent back after process is terminated". This is exactly the use case that Multiprocessing.Queue was designed for.

PyMOTW is a great resource for a whole load of Python modules, including Multiprocessing. Check it out here: https://pymotw.com/2/multiprocessing/communication.html

A simple example of how to send ongoing messages from a child to the parent using multiprocessing and loops:

import multiprocessing

def child_process(q):
    for i in range(10):
        q.put(i)
    q.put("done")  # tell the parent process we've finished

def parent_process():
    q = multiprocessing.Queue()
    child = multiprocessing.Process(target=child_process, args=(q,))
    child.start()
    while True:
        value = q.get()
        if value == "done":  # no more values from child process
            break
        print value
        # do other stuff, child will continue to run in separate process