Search code examples
pythonamazon-ecsaws-fargateamazon-cloudwatchlogspython-logging

python logging in AWS Fargate, datetime duplicated


I'm trying to use python logging module in AWS Fargate. The same application should work also locally, so I'd like to use a custom logger for local use but to keep intact cloudwatch logs. This is what I'm doing:

if logging.getLogger().hasHandlers():
    log = logging.getLogger()
    log.setLevel(logging.INFO)
else:
    from logging.handlers import RotatingFileHandler
    log = logging.getLogger('sm')
    log.root.setLevel(logging.INFO)
    ...

But I get this in cloudwatch:

2023-02-08T13:06:27.317+01:00   08/02/2023 12:06 - sm - INFO - Starting

And this locally:

08/02/2023 12:06 - sm - INFO - Starting

I thought Fargate was already defining a logger, but apparently the following has no effect:

logging.getLogger().hasHandlers()

Ideally this should be the desired log in cloudwatch:

2023-02-08T13:06:27.317+01:00   sm - INFO - Starting

Solution

  • Fargate just runs docker containers. It doesn't do any setup of your Python code that happens to be running in that docker container for you. It doesn't even know or care that you are running Python code.

    Anything written to STDOUT/STDERR by the primary process of the docker container gets sent to CloudWatch Logs, so if you want to be compatible with ECS CloudWatch Logs just make sure you are sending logs in the format you want to the console.