Terragrunt for_each value, can't retrieve data in other resources

Question

I have an issue with my terragrunt/terraform code as below. I don't know the right way to retrieve my both crawlers created by my for_each loop.

Normally I create it with for and count. I can't retrieve the correct values in my action triggers (main.tf).

terragrunt file (input):

inputs = {
  glue_crawler = {
      crawler    = {
        crawler_name     = "test",
        description      = "test crawler"
        },
      crawler1   = {
        crawler_name     = "test2",
        description      = "test2 crawler"
        }
    }
}

main.tf

#crawler declaration
resource "aws_glue_crawler" "default" {
  for_each = var.glue_crawler

  database_name = aws_glue_catalog_database.database.name
  name          = "Crawler_${each.value.crawler_name}"
  description   = each.value.description
  role          = aws_iam_role.svc-glue-crawler.id
  table_prefix  = "raw_"
  tags          = var.tags

  s3_target {
    path = "${var.s3_glue_name}/${each.value.crawler_name}"
  }

 configuration = jsonencode(var.crawler_configuration)
}

...

#trigger
resource "aws_glue_trigger" "my_trigger" {
  name         = var.trigger_name
  schedule     = "cron(00 01 * * ? *)"
  type         = "SCHEDULED"
  enabled      = "false"
  tags         = var.tags

  actions {
    job_name   = aws_glue_crawler.default[0].name
  }

  actions {
    job_name   = aws_glue_crawler.default[1].name
  }

variable.tf

variable "glue_crawler" {
  type = map(object({
    crawler_name = string
    description   = string
  }))
  default     = {}
  description = "glue crawler definitions."
}

When i run this code i have the following errors:

Error: Invalid index

  on main.tf line 294, in resource "aws_glue_trigger" "my_trigger":  294:     job_name   = aws_glue_crawler.default[0].name
    |----------------
    | aws_glue_crawler.default is object with 2 attributes

The given key does not identify an element in this collection value.


Error: Invalid index

  on main.tf line 298, in resource "aws_glue_trigger" "my_trigger":  298:     job_name   = aws_glue_crawler.default[1].name
    |----------------
    | aws_glue_crawler.default is object with 2 attributes

The given key does not identify an element in this collection value.

Answer 1

When you use for_each instead of count you need to access the specific element with the key and not the index. So this will be crawler and crawler1 instead of 0 and 1 in your example:

resource "aws_glue_crawler" "default" {
  for_each = var.glue_crawler

  database_name = aws_glue_catalog_database.database.name
  name          = "Crawler_${each.value.crawler_name}"
  description   = each.value.description
  role          = aws_iam_role.svc-glue-crawler.id
  table_prefix  = "raw_"
  tags          = var.tags

  s3_target {
    path = "${var.s3_glue_name}/${each.value.crawler_name}"
  }

 configuration = jsonencode(var.crawler_configuration)
}

...

#trigger
resource "aws_glue_trigger" "my_trigger" {
  name         = var.trigger_name
  schedule     = "cron(00 01 * * ? *)"
  type         = "SCHEDULED"
  enabled      = "false"
  tags         = var.tags

  actions {
    job_name   = aws_glue_crawler.default["crawler"].name
  }

  actions {
    job_name   = aws_glue_crawler.default["crawler1"].name
  }
}

But of course that only works that specific input. Instead you should consider making the actions parameter dynamic and using for_each over the crawlers here too:

resource "aws_glue_trigger" "my_trigger" {
  name         = var.trigger_name
  schedule     = "cron(00 01 * * ? *)"
  type         = "SCHEDULED"
  enabled      = "false"
  tags         = var.tags

  dynamic "actions" {
    for_each = aws_glue_crawler.default

    content {
      job_name = actions.name
    }
  }
}