Search code examples
pythonpython-3.xwindowsmultiprocessingcx-freeze

Multiprocessing python within frozen script

I am trying to compile a script utilizing multiprocessing into a Windows executable. At first I ran into the same issue as Why python executable opens new window instance when function by multiprocessing module is called on windows when I compiled it into an executable. Following the accepted answer I adjusted my script such that

from multiprocessing import freeze_support
# my functions
if __name__ == "__main__":
    freeze_support()
    # my script

And this again works perfectly when run as a script. However, when I compile and run it I encounter:

Error message

Where I've underlined in green part of the error. This specific line refers to

freeze_support()

in my script. Furthermore, it is not actually encountered on this line, but when my script goes to multiprocess which is something like:

p = multiprocessing.Process(target=my_function, args=[my_list])
p.start()
p1 = multiprocessing.Process(target=my_function, args=[my_list])
p1.start()
p.join()
p1.join()

Is this an error in the multiprocessing module (specifically line 148) or am I misunderstanding the answer I linked, or something else?

I'll also note that the script does work correctly when compiled, but you have to click "OK" on an error message for every multiprocess that is spawned (quite a lot) and every error message is exactly the same. Would this mean i am improperly ending the process with the p.join()?

I've also tried the solution at Python 3.4 multiprocessing does not work with py2exe which recommends adding

multiprocessing.set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe'))

to your script, yet this causes an error in the script form (not even compiled yet) of:

FileNotFoundError: [WinError 2] The system cannot find the file specified

Thanks for the help!

freeze_support documentation: https://docs.python.org/2/library/multiprocessing.html#multiprocessing.freeze_support

Solution

  • This appears to have been a problem for quite some time - I found references going back to 2014, at least. Since it appears to be harmless, the general recommendation is to suppress the error by replacing sys.stdout (and sys.stderr, which is flushed on the next line) with a dummy. Try this:

    import os
import sys
from multiprocessing import freeze_support

if __name__ == '__main__':
    if sys.stdout is None:
        sys.stdout = sys.stderr = open(os.devnull, 'w')
    freeze_support()