Search code examples
pythonsocketspipeipc

Interprocess communication in Python


What is a good way to communicate between two separate Python runtimes? Things tried:

  • reading/writing on named pipes e.g. os.mkfifo (feels hacky)
  • dbus services (worked on desktop, but too heavyweight for headless)
  • sockets (seems too low-level; surely there's a higher level module to use?)

My basic requirement is to be able to run python listen.py like a daemon, able to receive messages from python client.py. The client should just send a message to the existing process and terminate, with return code 0 for success and nonzero for failure (i.e. a two-way communication will be required.)


Solution

  • The multiprocessing library provides listeners and clients that wrap sockets and allow you to pass arbitrary python objects.

    Your server could listen to receive python objects:

    from multiprocessing.connection import Listener
    
    address = ('localhost', 6000)     # family is deduced to be 'AF_INET'
    listener = Listener(address, authkey=b'secret password')
    conn = listener.accept()
    print 'connection accepted from', listener.last_accepted
    while True:
        msg = conn.recv()
        # do something with msg
        if msg == 'close':
            conn.close()
            break
    listener.close()
    

    Your client could send commands as objects:

    from multiprocessing.connection import Client
    
    address = ('localhost', 6000)
    conn = Client(address, authkey=b'secret password')
    conn.send('close')
    # can also send arbitrary objects:
    # conn.send(['a', 2.5, None, int, sum])
    conn.close()