Search code examples
pythondockersubprocessstdoutstderr

How to prevent Docker from messing with subprocess output order? (MCVE included)

Consider this issue.py file:

import subprocess

print('Calling subprocess...')
subprocess.run(['python', '--version'])
print('Subprocess is done!')

Executing python issue.py manually yields what I expect:

Calling subprocess...
Python 3.9.0
Subprocess is done!

However, if I execute this inside a Docker container, something weird happens:

$ docker run --rm -v $(pwd):/issue python:3.9.0 python /issue/issue.py
Python 3.9.0
Calling subprocess...
Subprocess is done!

How can I fix this, to make Docker respect the correct output order?

Notes:

  • This problem also happens with stderr, although the above MCVE does not show it.
  • The MCVE uses the python image directly but in my real use case I have a custom image from a custom Dockerfile which uses FROM python.
  • Using capture_output=True in the subprocess.run call and then printing the captured output is not an option for me because my real use case invokes a subprocess that prints information over time to stdout (unlike python --version) and I cannot wait for it to complete to print the entire output only after that.
Solution

  • As @DavidMaze pointed out in a comment, I just needed to set the PYTHONUNBUFFERED environment variable to 1. This can be done for example with:

    docker run --rm -e PYTHONUNBUFFERED=1 -v $(pwd):/issue python:3.9.0 python /issue/issue.py