Search code examples
pythonlogginggunicorn

Logging from multiple workers in Gunicorn


I'm working on a backend microservice, using the built in python logging library to monitor what is happening in my Flask app, and using Gunicorn to spin up multiple workers with it.

Ideally I would like to create a parent logger (logging.GetLogger('Service')), and each worker would use a child logger (logging.GetLogger('Service.Worker1'), logging.GetLogger('Service.Worker2'), etc.). However, I'm not sure if I have a way to differentiate worker processes.

Can I pass somehow different arguments for each new gunicorn worker?

Right now when I'm using the same logger in each worker (logging.GetLogger('Service') for all), I have no way to tell which worker created the log. This might not be an issue, but if I'm using a logging.FileHandler then separate processes may collide.


Solution

  • One workaround that I came up with is simply using os.getpid(), which will be unique in each worker. While this does the job and allows me to create unique names for logging, I'm still wondering if there is a better way.