Search code examples
pythonmultiprocessingnamed-pipes

Does multiprocessing support named pipes (FIFO)?


Multiprocessing's Pipes and Queue are based on anonymous pipes, does the multiprocessing of Python provide named pipes (FIFO)?


Solution

  • There's no built-in support for a cross-platform abstraction of named pipes in multiprocessing.

    If you only care about Unix, or only about Windows, you can of course create named pipes manually. For Unix, mkfifo is in the stdlib. For Windows, you have to use ctypes or cffi, or a third-party library like win32api to call CreateFile with the right arguments.

    Trying to abstract over the semantic differences between the two is pretty painful, which is probably why the stdlib doesn't attempt to do so. (For example, Windows named pipes are volatile; posix named pipes are permanent.)

    Here's a trivial Unix example:

    import multiprocessing
    import os
    
    def child():
        with open('mypipe', 'rb') as p:
            print(p.read())
    
    def main():
        try:
            os.mkfifo('mypipe')
        except FileExistsError:
            pass
        multiprocessing.Process(target=child).start()
        with open('mypipe', 'wb') as p:
            p.write(b'hi')
        os.remove('mypipe')
    
    if __name__ == '__main__':
        main()