Timeout in I/O function call in child thread

Question

I have a Flask-RESTful API (Python 3) and I'm trying to timeout a function call that runs a terminal command which executes a C++ program which may take a long time to complete. This function looks something like this:

def func(args):
    ...
    result = subprocess.getoutput('./c_program')
    ...

I searched how to do this and I came across the following: Timeout on a function call I tried both of the suggested methods, but unfortunately neither work. Signal only works in the main thread (and I am in a threaded API), and multiprocessing doesn't stop I/O work. I kept looking but I only found mention of these two methods. Does anyone know any solution to this?

Answer 1

Better use Popen from the subprocess module. I also included the strategy from How to terminate a python subprocess launched with shell=True to fix issues with killing the process:

import os
import signal
import subprocess
import time
from subprocess import PIPE, Popen

p = Popen("sleep 1; echo 1", shell=True, stdout=PIPE, preexec_fn=os.setsid)

WAIT = 0.5

started = time.time()

# do some other work for WAIT seconds:
while time.time() < started + WAIT:
    print("do something else")
    time.sleep(.1)

print("kill")

try:
    os.killpg(
        os.getpgid(p.pid), signal.SIGTERM
    )
except ProcessLookupError:
    # process possibly died already
    pass

print(p.stdout.read())

You can set WAIT to 1.5 to let the process complete, in this case you will see the output from echo.