Search code examples
pythondockertqdm

tqdm progress bar with Docker logs

I am using tqdm to display various progress bars for my Python console application. For the production deployment of the applications, I use Docker.

The progress bars work fine when running a Python application in a terminal. However, when Dockerized and the terminal output is accessed through docker logs the progress bar does not function because as far as I understand it is not an interactive terminal. Although it looks like the progress gets rendered if docker logs is dumped after the progress bar have completed, but not sure if there are some other conditions for this to happen (output buffering?).

I would like to modify my tqdm behavior so that

  • It detects when it is run in non-interactive Dockerised environment
  • Instead of displaying interactive progress bar, it will log completion statements (10% done, X iterations/s) regularly

This way the progress durations and such would be more accessible when running the application in production.

What would be the way to attach such a custom behavior to tqdm?

Solution

  • The package tqdm_loggable is a drop in replacement for tqdm that works well for this use case.

    To install:

    pip install tqdm-loggable

    Then just replace any imports of tqdm (from tqdm import tqdm) with:

    from tqdm_loggable.auto import tqdm

    Be sure to set logging level to INFO to see the results in the logs:

    import logging

logging.basicConfig(level=logging.INFO)