Terraform - nested loop for vary attributes number

Question

Description

I'm having the following input structure (*.tfvars.json)

{
   "projects":[
      {
         "name":"Project 1",
         "gitlab":{
            "variables":[
               {
                  "name":"Variable 1",
                  "value":"Value 1"
               }
            ]
         }
      }
   ]
}

Parsed to a following tf variable:

variable "projects" {
  type = list(
  object({
    name              = string
    gitlab            = object({
      variables = list(
      object({
        name  = string
        value = string
      })
      )
    })
}

the number of projects, as well as the number of variables in each project, can vary, therefore defined as lists.

First I'm creating the projects (gitlab provider) with a simple "count" meta-parameter:

resource "gitlab_project" "projects" {
  count    = length(var.projects)
  name     = var.projects[count.index].name
  (...)
}

but then I'm struggling to create the projects' variables.

Issue

Using "count" won't work as it does not support nesting;
so the other approach I've found was to go with for-each loop.
I've flattened the map:

locals {
  project_variables = flatten([
     for project in var.projects : [
        for variable in project.gitlab.variables : {
           project_name = project.name
           variable     = variable
        }
     ]
  ])
}

but it still gives me nothing, as I don't see an option to retrieve the created project ID using unique attributes (like the name), and the project ID is required to create the variable resource:

resource "gitlab_project_variable" "project_variables" {
  project = gitlab_project.projects[...].id
  key     = ""
  value   = ""
}

It does seem like a reasonably simple requirement (just a nested loop that uses the first loop iterator as the index to retrieve project ID), but the solutions I'm finding are either not covering this particular case (with an undefined/variable number of arguments in loops), or are overly complicated...

Could you guys share how would you approach such a problem with terraform-newbie?

Answer 1

You are correct, you have to flatten your projects, but in a different way:

variable "projects" {
  type = list(
  object({
    name              = string
    gitlab            = object({
      variables = list(
      object({
        name  = string
        value = string
      })
      )
    })
  }))  
    
  default =   [
      {
         "name":"Project 1",
         "gitlab":{
            "variables":[
               {
                  "name":"Variable 1",
                  "value":"Value 1"
               }
            ]
         }
      },
      {
         "name":"Project 2",
         "gitlab":{
            "variables":[
               {
                  "name":"Variable 2",
                  "value":"Value 2"
               },
               {
                  "name":"Variable 3",
                  "value":"Value 3"
               }               
            ]
         }
      },      
      
   ]
}



locals {

  project_names = distinct([for project in var.projects: project.name])

  project_variables = merge([
          for project in var.projects:
            {
                for variable in project["gitlab"]["variables"]:
                 "${project.name}-${variable.name}" => {
                     project_name = project["name"]
                     var_name = variable.name
                     var_value = variable.value
                 }
            }
      ]...) # do NOT remove the dots
}

which will give:

project_variables = {
  "Project 1-Variable 1" = {
    "project_name" = "Project 1"
    "var_name" = "Variable 1"
    "var_value" = "Value 1"
  }
  "Project 2-Variable 2" = {
    "project_name" = "Project 2"
    "var_name" = "Variable 2"
    "var_value" = "Value 2"
  }
  "Project 2-Variable 3" = {
    "project_name" = "Project 2"
    "var_name" = "Variable 3"
    "var_value" = "Value 3"
  }

then:

resource "gitlab_project" "projects" {
  for_each    = toset(local.project_names)
  name        = each.key
  (...)
}

resource "gitlab_project_variable" "project_variables" {
  
  for_each = local.project_variables
  
  project = gitlab_project.projects[each.value.project_name].id
  key     = each.value.var_name
  value   = each.value.var_value
}