Search code examples
pythonbashbufferstdoutstderr

Python vs bash stderr stdout buffering


How can I make my python script behave like bash with regards to stderr and stdout buffering? Bash is buffering stderr and stdout so that printed messages appear in chronological order. The example scripts below illustrate the behavior. Tested with Python 2.7 on Ubuntu 14.04.

cstderr.c

#include <stdio.h>
#include <stdlib.h>
main()
{
    fprintf(stderr, "C out to stderr\n");
    exit(0);
}

bash_stderr.sh

echo Before C cstderr
./cstderr.out
echo After C cstderr

py_stderr.py

#!/usr/bin/env python

import subprocess
print("Before C cstderr")
subprocess.check_call("./cstderr.out")
print("After C cstderr")

Bash behavior

$ ./bash_stderr.sh > outfile 2>&1
$ cat outfile
Before C cstderr
C out to stderr
After C cstderr

Python behavior

$ ./py_stderr.py > outfile 2>&1
$ cat outfile 
C out to stderr
Before C cstderr
After C cstderr

Solution

  • Bash flushes stdout before any executing any external program. Python doesn't. To get the desired behavior, call sys.stdout.flush() immediately before the subprocess.check_call().