How to wait until multiprocessing.connection.Client is available in python?

Question

I have a process.py script:

from multiprocessing.connection import Listener

with Listener(('localhost', 6000)) as listener:
    with listener.accept() as connection:
        message = connection.recv()
        # [...]
        connection.send(message)

I launch this script in a thread (to avoid blocking the main thread) with:

import threading, subprocess
threading.Thread(target=lambda: subprocess.run(['python', 'process.py'])).start()

But sometimes I want to wait (block the main thread) until my process has launched.

Here is what I do:

from multiprocessing.connection import Client

nAttempts = 0
while True:
    try:
        with Client(('localhost', 6000)) as connection:
            connection.send(nAttempts)
            message = connection.recv()
            # [...]
            break
    except ConnectionRefusedError:
        nAttempts += 1
        pass

Is there a better approach?

The program tries ~282 connections before being able to connect, can it be a problem?

Answer 1

First, modify process.py so that you can import what you need:

from multiprocessing.connection import Listener

def listen(listening_event=None):
    with Listener(('localhost', 6000)) as listener:
        if listening_event:
            listening_event.set()  # Signal that we are now listening
    
        with listener.accept() as connection:
            message = connection.recv()
            # [...]
            connection.send(message)

if __name__ == '__main__':
    listen()

Then you main script would be:

import threading

from process import listen

listening_event = threading.Event()
t = threading.Thread(target=listen, args=(listening_event,))
t.start()
listening_event.wait()  # Wait for this notification
...
t.join()  #  Wait for thread to terminate (if it ever does)

If you need to use multiprocessing, then:

import multiprocessing

from process import listen

if __name__ == '__main__':
    listening_event = multiprocessing.Event()
    p = multiprocessing.Process(target=listen, args=(listening_event,))
    p.start()
    listening_event.wait()  # Wait for this notification
    ...
    p.join()  #  Wait for process to terminate (if it ever does)

Update

Based on your comment, we can determine that the process.py listener is ready for accepting connection in the following way:

The Main Script

from multiprocessing.connection import Listener, Client
from threading import Thread
import subprocess

def run_process():
    subprocess.run('conda activate myenv && python process.py', shell=True)

def wait_for_listener_ready():
    with Listener(('localhost', 6001), backlog=5) as listener:
        process_thread = Thread(target=run_process)
        process_thread.start()
        with listener.accept() as connection:
            connection.recv()
            connection.send('Okay')
    return process_thread

# Wait process.py to be ready:
process_thread = wait_for_listener_ready()

...

process_thread.join()

Here the main script is starting its own listener on an agreed-up port (assumed to be 6001). The process.py script will send a message to our listener when it is ready to accept connections:

process.py

from multiprocessing.connection import Listener, Client

with Listener(('localhost', 6000), backlog=5) as listener:
    with Client(('localhost', 6001)) as starter_connection:
        starter_connection.send('Hello')
        starter_connection.recv()

    with listener.accept() as connection:
        message = connection.recv()
        connection.send(message.upper())