Python fork process inside Jupyter Notebook

Question

I am running the following code inside a Jupyter notebook:

import os

print("Start")
pid = os.fork()

if pid == 0:
    print("Child")
    os._exit(os.EX_OK)
else:
    print("Parent")
    
if pid != 0:
    # parent
    pid, status = os.waitpid(pid, 0)
    print("Done")

I am getting the following output "almost" every time:

Child
Start
Parent
Done

How is it that "Child" gets printed before "Start"? Almost 9 out of 10 times, I get the output as above. Occasionally, I find what is intuitively expected ("Start" being printed first, followed by "Parent" or "Child" and then finally ending with "Done").

When I run the same code directly on the console, I get the expected outcome each time:

Start
Parent
Child
Done

Why do we see this peculiar behavior within Jupyter notebooks? And how to avoid this?

Answer 1

It looks like stdout is block buffered, not line buffered. The parent "Start\n" is waiting in the output buffer. The child "Child\n" starts out in its own output buffer but is flushed on exit. You could verify with import sys;print(sys.stdout.isatty()). The solution is to flush often

print("Start", flush=True)

or if you have multiple things to print,

print("Foo")
print("Bar")
sys.stdout.flush()