Search code examples
pythonpython-3.xmultiprocessingpython-multiprocessing

How to correctly terminate multiprocessing.Process()?


I've a code that needs to run multiple parallel threads in the following way :

def driver():
    #Gets a UID and passes it to Run
    run(uid)

def f1(a,b,c):
    #Does Some thing : Trivial Example Below - These Operations might take long time
    d=a+b+c
    print(d)

def f2(a,b,c):
    #Does Some thing : Trivial Example Below - These Operations might take long time
    d=a+b*c
    e=a+b/c
    print(d,e)

Now, I have a run() method that runs these in parallel.

from multiprocessing import Process
def run(uid):
    #Get a,b,c on the basis of UID then call the following
    thread_1=Process(target=f1, args=(a,b,c))
    thread_2=Process(target=f2, args=(a,b,c))
    thread_1.start()
    thread_2.start()
    thread_1.join()
    thread_2.join()

The entry point of the code is driver(). However, my code gets stuck after certain number of operations and I guess it is due to a memory issue.

My question is: How can I terminate the threads correctly after their successful execution? Do they still reside in memory after execution?

Is there anything wrong in the way I'm running these threads?


Solution

  • Use Threadpool executor or Threading library to manage things easier.

    Threadpool executor can be used as follows.

    def test(n):
     print(n)
                
    with ThreadPoolExecutor(max_workers=5) as executor:
      for i in range(10):
         executor.submit(test, args)
    

    https://apscheduler.readthedocs.io/en/latest/userguide.html