Piping input into running Python script

Question

Maybe this is the wrong approach, so feel free to redirect.

I am trying to asynchronously pipe input into a python script. The idea is that I have a background Python script do some long initial setup functions then wait for input. Once input is received, the script will perform some more functions and return to waiting for input. This will not be on a headless server, so this is not "user input"

Here is my script:

import sys

# do some long functions

sys.stdin.flush()

data=""

while(1)
    data = sys.stdin.read()
    if data != "":
        print("input: " + data)
        sys.stdin.flush()
    data=""
exit()

I am on Ubuntu 22.04. In one terminal I run the script with the following command: python3 rx.py</tmp/py-srv.input & echo $!>/tmp/py-srv.pid this creates a pid file so I can kill it later and an input file I can write to to input into the script.

In another terminal I write to the input file:echo "test">/tmp/py-srv.input

and in the first terminal I see: input: test Yay! it works....

But if I try to write to the input file again echo "YAY!">/tmp/py-srv.input I get nothing on the first terminal.

Interinstingly though, if the second time I write somehting like echo "testYAY!">/tmp/py-srv.input I get input: AY! in the first terminal.

I looked at the .input file and I noticed that it is cleared after the first input is read, but not after subsequent reads. This is what led to adding sys.stdin.flush(), but that didn't solve anything.

I have tried using the input() function but I get an EOFError: EOF when reading a line error which I assume is from launching the script as a background process with the single &

Its seems like I am close, but it does not work with multiple input writes. Am I missing something with python's sys.stdin? Why doesn't stdin.read() see the second write to the .input file unless it starts with the string from the first write?

Any help would be great, thank you.

Answer 1

I think it's better to use a FIFO rather than a regular file to communicate with a server like this. Here's a brief example of what this would look like:

Server:

import os

os.mkfifo('py-srv.fifo')

while True:
    with open('py-srv.fifo') as f:
        data = f.read()
        print('input:', data)

Client:

with open('py-srv.fifo', 'w') as f:
    f.write('hello')