Why does AWS ECS FARGATE not generate log on cloud watch (terraform)?

Question

(I replace sensitive information with XXXXXX, YYYYYY)

1. containerized app

test.py:

import logging
logging.info("wow!!")

Dockerfile:

FROM python:3.10-slim
RUN apt-get update && \
    apt-get -y install git && \
    rm -rf /var/lib/apt/lists/*
COPY test.py /tmp/test.py
CMD ["python", "/tmp/test.py"]

2. Push image on ecr

$ docker push XXXXXXX.dkr.ecr.us-east-1.amazonaws.com/chois-trader:trading-latest

3. terraform

ecs.tf:

resource "aws_kms_key" "chois_trader" {
  deletion_window_in_days = 7
}

resource "aws_cloudwatch_log_group" "chois_trader" {
  name = "chois_trader"
}



resource "aws_iam_role" "chois_trader_task_execution_role" {
  name = "chois-trader-task-execution-role"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": [
        "sts:AssumeRole"
      ]
    }
  ]
}
EOF
}

resource "aws_iam_role_policy_attachment" "chois_trader_task_execution_role_policy_attachment" {
  role       = aws_iam_role.chois_trader_task_execution_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy"
}





resource "aws_iam_role" "chois_trader_task_role" {
  name = "chois-trader-task-role"

  assume_role_policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "Service": "ecs-tasks.amazonaws.com"
        },
        "Action": "sts:AssumeRole"
      }
    ]
  })
}

resource "aws_iam_policy" "chois_trader_log_policy" {
  name        = "chois-trader-log-policy"
  description = "chois trader log IAM policy"
  policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "1",
        "Effect": "Allow",
        "Action": [
          "logs:CreateLogStream",
          "logs:CreateLogGroup",
          "logs:DescribeLogStreams",
          "logs:PutLogEvents"
        ],
        "Resource": "arn:aws:logs:ap-northeast-2:YYYYYYYYY:log-group:chois_trader:*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "example_attachment" {
  role       = aws_iam_role.chois_trader_task_role.name
  policy_arn = aws_iam_policy.chois_trader_log_policy.arn
}




resource "aws_ecs_cluster" "chois_trader" {
  name = "chois_trader"

  configuration {
    execute_command_configuration {
      kms_key_id = aws_kms_key.chois_trader.arn
      logging    = "OVERRIDE"

      log_configuration {
        cloud_watch_encryption_enabled = true
        cloud_watch_log_group_name     = aws_cloudwatch_log_group.chois_trader.name
      }
    }
  }
}

# Create a task definition with a container image
resource "aws_ecs_task_definition" "chois_trader_task" {
  family = "chois-trader-task"
  requires_compatibilities = ["FARGATE"]
  network_mode             = "awsvpc"
  execution_role_arn       = aws_iam_role.chois_trader_task_execution_role.arn
  task_role_arn = aws_iam_role.chois_trader_task_role.arn
  cpu    = 256
  memory = 512
  container_definitions = jsonencode([
    {
      name  = "chois-trader-task-container"
      image = "XXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/chois-trader:trading-latest"
      log_configuration = {
        log_driver = "awslogs"
        options = {
          "awslogs-group"         = "chois_trader"
          "awslogs-region"        = "ap-northeast-2"
          "awslogs-stream-prefix" = "ecs"
        }
      }
    }
  ])
  volume {
    name      = "service-storage"
  }
}

4. tf apply

$ terraform apply

4. Execute task (AWS console)

After a few seconds, exit with status code 0!

But log not appear at all..

5. Reference

• https://www.chakray.com/creating-fargate-ecs-task-aws-using-terraform/
• Terraform AWS CloudWatch log group for ECS tasks/containers

Anything i missed?

Answer 1

Below is our ECS task definition template through which the logs are going through fine. I am guessing there may be issue due to usage of underscore_separated keys instead of camelCase.

"logConfiguration": {
    "logDriver": "awslogs",
    "options": {
      "awslogs-group": "${cw_log_group_name}",
      "awslogs-region": "${aws_region}",
      "awslogs-stream-prefix": "${app_name}-${environment}-log-stream"
    }
},