python: fork without an external command, and capturing stdout and stderr separately

Question

I'd like to fork a subprocess in python that does not run an external command ... it would just run a defined function. And I want to capture stdout and stderr separately.

I know how to use os.fork() and os.pipe(), but that mechanism only gives me two fd's to work with. I'm looking for three fd's: one for stdin, one for stdout, and one for stderr. This is easy to manage using subprocess.Popen when running an external command, but that function doesn't seem to allow a local function to be forked; only a separate executable.

In ruby, the popen3 command can take "-" as its command argument, and in this case, a fork takes place without any external command being invoked, and the 3 fd's I mentioned are returned. Is there some sort of python analog to this routine in python?

Answer 1

• If you want to redirect the stdout and stderr separately from the child process, you can simply create two separate pipes for each, instead of one. I have shared the relevant code.

• You can also read this thread to gain more knowledge on this subject: Redirect stdout to a file in Python?

• I have mentioned two methods for writing to stdout, and stderr from the child process (Method1, Method2)

• If you want to write to stdin of child process as well, you should create another file descriptor. This time the r would go to child process, and the w would go to the parent process.

import os
import sys
import time

# Create two pipes. One for sys.stdout, and one for sys.stderr
r_out, w_out = os.pipe()
r_err, w_err = os.pipe()

pid = os.fork()
if pid == 0:
    # Child process
    os.close(r_out)
    os.close(r_err)

    w1 = os.fdopen(w_out, "w")
    w2 = os.fdopen(w_err, "w")
    sys.stdout = w1
    sys.stderr = w2

    # Note that flush=True is necessary only if you want to ensure the order of messages printed
    # across method1, and method2 is maintained

    # Method 1: Standard Python print messages
    print("Redirected to stdout #2", flush=True)
    print("Redirected to stderr #2", file=sys.stderr, flush=True)

    # Method 2: Using system file descriptors
    stdout_fd = sys.stdout.fileno()
    os.write(stdout_fd, b'Redirected to stdout')

    stderr_fd = sys.stderr.fileno()
    os.write(stderr_fd, b'Redirected to stderr')

    # Restore original stdout, and stderr
    sys.stdout = sys.__stdout__
    sys.stderr = sys.__stderr__

    # Close the file descriptors
    w1.close()
    w2.close()

else:
    # Parent process
    os.close(w_out)
    os.close(w_err)

    r1 = os.fdopen(r_out)
    r2 = os.fdopen(r_err)
    for i in range(5):
        # Note that r1.read(), and r2.read() are non-blocking calls
        # You can run this while loop as long as you want.
        print("Read text (sysout):", r1.read())
        print("Read text (syserr):", r2.read())
        time.sleep(0.5)