Disable ECS container logs - specifically Airflow scheduler logs that are taking up disk space

Question

Is there way to prevent logs from writing in an ECS task, for us this is happening for the Airflow scheduler, we keep seeing disk space full. Is there a way to disable logs so this doesn't continue happening?

Answer 1

You can reduce the number of logs by setting log level in the service level, ECS has nothing to do with the application logs.

AIRFLOW__LOGGING__LOGGING_LEVEL="CRITICAL"

airflow logging-level

Or you can also try

log-cleanup A maintenance workflow that you can deploy into Airflow to periodically clean out the task logs to avoid those getting too big.

airflow-maintenance-dags

The other recommended option is to write logs to cloud watch, and then you can set retention whatever suit you. also it will not fill the ec2 space as it will use aws log driver, not the docker one

{
    "containerDefinitions": [
        {
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "airflow",
                    "awslogs-region": "us-west-2",
                    "awslogs-create-group": "true",
                    "awslogs-stream-prefix": "airflow"
                }
            }
}

AWS log driver

It will incur costs as well, so better to check the cost too.

Collect (Data Ingestion)    $0.50 per GB
Store (Archival)    $0.03 per GB

https://aws.amazon.com/cloudwatch/pricing/