Fix time in timeout of multiprocessing process

Question

I was trying to set up a code of parallel jobs using multiprocessing module. I would like to terminate all the child job within 10 seconds. I made the following code using timeout argument in process join function. However, I found the termination time of all the jobs depend on both the values of the timeout and the number of jobs. How do I calculate the accurate time?

import time
from multiprocessing import Manager, Process

def f(x, dic):
    time.sleep(60)
    print "Time: ", x
    dic[x] = "Done"

mng = Manager()
dic = mng.dict()
jobs = [Process(target=f, args=(i, dic)) for i in range(50)]

for job in jobs:
    job.start()

for job in jobs:
    job.join(timeout=10)

for job in jobs:
    job.terminate()

for job in jobs:
    job.join()

print dic.keys()

Answer 1

You can calculate the ten second timeout first, then wait and terminate jobs in a single loop. You only wait if the timeout hasn't arrived.

end_at = time.time() + 10
while jobs:
    job = jobs.pop()
    delta = end_at - time.time()
    if delta > 0:
        job.join(timeout=delta)
    job.terminate()
    job.join()