I have a working celery app. I want to add structured logging to it.
A complete working example would be hard to provide, so let me demostrate:
import structlog
import logging
logging.config.dictConfig(...)
structlog.configure(...)
app = Celery()
Then, in some task:
logger = structlog.getLogger()
logger.info("Foo")
In this app, when there are logs outside Celery tasks, they print out in a JSON format - like I want. However, each log from a Celery task has this:
[2024-04-25 20:37:14,305: INFO/ForkPoolWorker-2] {...}
[2024-04-25 20:37:14,305: INFO/MainProcess] {...}
prefix. And those logs don't have keys that should be here (due to my logging configuration), so that suggests that the logging isn't "configured" inside Celery tasks.
My question is: how do I make Celery tasks log how I want them to? Could I maybe move those two logging.config.dictConfig and structlog.configure calls to a separate function (let's say initialize_logging) and add it as a parameter or something to Celery so it executes it at the beggining of every task or something like this?
I've rushed this question, no doubt. There's an easy answer, Celery provides a signal for this exact purpose: setup_logging