How would I handle the event of a thread finishing in python?

Question

Say I am parsing data from a few different exchanges, I want each extra to run simultaneously so I start each one up in its own process, but inside each process I want to add output string to a list then return that list to the main and output it to a curses UI. Simple version:

def exchange1():
    #do stuff
    #add to output list
    return output_list

def exchange2():
    #do stuff
    #add to output list
    return output_list

output_list = startnewprocess(exchange1)
output_list = startnewprocess(exchange2)

window.getch()
#have it so I can input different options to do stuff while those threads above are running

how do I make it so that when one of the processes finishes it starts back up again?

OR

Alternatively, how do I make it so that I can retrive what has been added to output_list within the process from the main function so that I can just have a while True: loop inside the exchange functions, so I can output the data to the screen outside of the processes?

Answer 1

You can use multiprocessing to do this:

import time
import multiprocessing

def exchange1(q):
    while True:
        time.sleep(2)
        q.put([1, 2, 3, 4, 5])

def exchange2(q):
    while True:
        time.sleep(4)
        q.put([4, 5, 6, 7, 8])

if __name__ == "__main__":
    q = multiprocessing.Queue()
    p1 = multiprocessing.Process(target=exchange1, args=(q,))
    p2 = multiprocessing.Process(target=exchange2, args=(q,))
    p1.start()
    p2.start()
    while True:
        out = q.get()
        print("out is {}".format(out))

Output:

out is [1, 2, 3, 4, 5]
out is [1, 2, 3, 4, 5]
out is [4, 5, 6, 7, 8]
out is [1, 2, 3, 4, 5]
out is [1, 2, 3, 4, 5]
out is [4, 5, 6, 7, 8]

Note that if you also want to use getch to read characters in, you're going to have to listen for data coming back from the queue in a separate thread in the parent process, and use some thread-safe mechanism in the curses library to update the UI:

def waiter(q):
    while True:
        out = q.get()
        print("out is {}".format(out))
        # Update UI in a thread-safe way

if __name__ == "__main__":
    q = multiprocessing.Queue()
    p1 = multiprocessing.Process(target=exchange1, args=(q,))
    p2 = multiprocessing.Process(target=exchange2, args=(q,))
    p1.start()
    p2.start()
    t = threading.Thread(target=waiter, args=(q,))
    t.daemon = True
    t.start()
    while True:
        char = window.getch()
        # Other stuff