Search code examples
pythonparallel-processingargumentscx-oracle

Launch 3 distinct functions with arguments at the same time


I would like to start 3 distinct functions at the same time on machine with multiple cores. Each function launches an extensive data query to separate databases using the same module (cx_Oracle) and each function is called with some arguments. How could this be accomplished?

I have read a bit on parallel computing with Python but ProcessPoolExecutor always returns BrokenProcessPool error. The script is saved (reported as an issue in other SE item).

if __name__ == '__main__'
    with ProcessPoolExecutor(3) as executor:
        future_1 = executor.submit(query_1, arg1)
        future_2 = executor.submit(query_2, arg2)
        future_3 = executor.submit(query_3, arg3)
    result_1 = future_1.result()
    result_2 = future_2.result()
    result_3 = future_3.result()

BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Have you seen something similar? A reference to other thread or an example of three distinct calls with arguments is enough for me. I have also tried multiprocessing but not successfully.


Solution

  • The cx_Oracle sample Threads.py shows running two statements in different threads:

    . . .
    def TheLongQuery():
         . . .
    
    def DoALock():
        . . .
    
    
    thread1 = threading.Thread(None, TheLongQuery)
    thread1.start()
    
    thread2 = threading.Thread(None, DoALock)
    thread2.start()
    
    thread1.join()
    thread2.join()