Packaging python modules having MultiProcessing Code

Question

I am trying to package my python project into an executable using pyinstaller. The main module contains code for multiprocessing. When I run the executable, only the lines of code prior to the multi processing part get executed again and again. Neither does it throw an exception or exit the program.

Code in main module:

from Framework.ExcelUtility import ExcelUtility
from Framework.TestRunner import TestRunner
import concurrent.futures


class Initiator:

def __init__(self):
    self.exec_config_dict = {}
    self.test_list = []
    self.test_names = []
    self.current_test_set = []

def set_first_execution_order(self):
    # Code


def set_subsequent_execution_order(self):
    # Code

def kick_off_tests(self):
    '''Method to do Multi process execution'''
    if(__name__=="__main__"):
        with concurrent.futures.ProcessPoolExecutor(max_workers=int(self.exec_config_dict.get('Parallel'))) as executor:
            for test in self.current_test_set:
                executor.submit(TestRunner().runner,test)  ***This line is not being executed from the exe file.


initiator = Initiator()
initiator.get_run_info()
initiator.set_first_execution_order()
initiator.kick_off_tests()
while len(initiator.test_list) > 0:
    initiator.set_subsequent_execution_order()
    try:
        initiator.kick_off_tests()
    except BaseException as exception:
        print(exception)

Answer 1

From the problem definition I'm assuming you are using ms-windows, and that the main module is not named __main__.py.

In that case, multiprocessing has some special guidelines:

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).

and

Instead one should protect the “entry point” of the program by using if __name__ == '__main__'

So, change the last part of your main module like this:

from multiprocessing import freeze_support

def kick_off_tests(self):
    '''Method to do Multi process execution'''
        with concurrent.futures.ProcessPoolExecutor(max_workers=int(self.exec_config_dict.get('Parallel'))) as executor:
            for test in self.current_test_set:
                executor.submit(TestRunner().runner,test)  


if __name__ == '__main__':
    freeze_support()
    initiator = Initiator()
    initiator.get_run_info()
    initiator.set_first_execution_order()
    initiator.kick_off_tests()
    while len(initiator.test_list) > 0:
        initiator.set_subsequent_execution_order()
        try:
            initiator.kick_off_tests()
        except BaseException as exception:
            print(exception)