How to run two functions in parallel in python when one collects data from the other?

Question

I'm new to multiprocessing and want to collect data in one function and write data in another function simultaneously. Here's a psuedocode of what I have.

def Read_Data():
 for num in range(0,5):

     ## Read 5  random values

     print('Reading ' + str(values[num]))
 return(values)

def Write_data(values):

 ## Arrange the random values in ascending order

 for num in range(0,5):
     print('Writing' + str(arranged_values[num]))

if __name__=='__main__'
  values = Read_Data()
  Write_data(values)

I want the output to look like this.

reading 1, writing none
reading 3, writing none
reading 5, writing none
reading 4, writing none
reading 2, writing none
reading 7, writing 1
reading 8, writing 2
reading 10, writing 3
reading 9, writing 4
reading 6, writing 5

Now the reason I want it to run parallel is to make sure I'm collecting data all the time and not losing data while I'm modifying and printing.

How can I do it using multiprocessing?

Answer 1

This should illustrate a few concepts. The queue is used to pass objects between the processes.

The reader simply gets its value somewhere and places it on the queue.

The writer listens on the queue forever. Adding a "TERMINATED" signal is a very simple way of telling the writer to stop listening forever (there are other more effective ways using signals and events but this just illustrates the concept).

At the end we "join" on the two processes to make sure they exit before we exit the main process (otherwise they are left hanging in space and time)

from multiprocessing import Process, Queue
from time import sleep

def reader(q):
    for i in range(10):
        print("Reading", i)
        q.put(i)
        sleep(1)
    print("Reading TERMINATED")
    q.put("TERMINATE")


def writer(q):
    while True:
        i = q.get()
        if i == "TERMINATE":
            print("Writer TERMINATED")
            break
        print("Writing", i)

q = Queue()

pr = Process(target=reader, args=(q,))

pw = Process(target=writer, args=(q,))

pw.start()

pr.start()

pw.join()
pr.join()