Pathos multiprocessing pipe and queue on windows

Question

I'm trying to use the pathos library to replace the builtin multiprocessing library, but am having difficulty using either pipes or queues on windows. Here's a representative example:

from pathos.helpers import mp
#import multiprocessing as mp

def f(pipe, queue):
    if sys.gettrace():
        print 'Debug mode. About to crash'
    else:
        print 'Execute mode'
    pipe.send('pipe')
    queue.put('queue')


if __name__ == '__main__':
    mp.freeze_support()

    to_child, to_self = mp.Pipe()
    queue = mp.Queue()
    p = mp.Process(target=f, args=(to_child, queue))
    p.start()
    p.join()

pipe.send('pipe') raises IOError: (6, 'The handle is invalid') and queue.put('queue') raises WindowsError: [Error 5] Access is denied. Both work correctly using the vanilla multiprocessing module.

Am I doing something wrong?

Edit: This crash only occurs when I'm trying to debug child processes (I use WingIDE). I can accurately predict the crash by checking sys.gettrace(), as above.

Answer 1

It turns out the issue arises when Wing IDE is set to debug child processes. As I understand it, wing enables child process debugging by inserting itself between parent and child processes (which means that 'child' processes are now actually the grandchildren of the parent process). On windows, this is made possible by monkeypatching multiprocessing to cause Popen to set the 'inheritable' flag to True when calling duplicate() on handles.

Without the inheritable setting, grandchild processes cannot access the handles, and hence the exceptions above are raised.

It seems likely that a similar monkey patch could be applied to pathos's helpers.mp module to allow wing to debug child processes with pathos.