Dynamically create JSON template in Terraform

Trying to dynamically create a json object that I can pass to another module in Terraform, however this is proving to be difficult and/or may not be feasible.

I am fetching a list of IAM Roles, via

data "aws_iam_roles" "matching_roles" {
  for_each   = toset(["role-prefix1-.*", "ecs-role.*"])
  name_regex = each.value
}

This provides a map which includes a section of arns. I then collect all the arns into a list via

locals {
  flat_arns = flatten([
    for key, role_set in data.aws_iam_roles.matching_roles_2 : role_set.arns
  ])
}

Next I want to then encode this list into a json template. I have tried using the following


data "template_file" "dynamic_roles_os" {
  template = <<EOF
{
  "roles": [
    {% for role in roles %}
    {
      "role_name": "all_access",
      "role_mapping_payload": {
        "backend_roles": [
          "${role}"
        ],
        "hosts": [],
        "users": []
      }
    }{% if not loop.last %},{% endif %}
    {% endfor %}
  ]
}
EOF

  vars = {
    roles = local.flat_arns
  }
}

resource "local_file" "backend_roles_json" {
  content  = data.template_file.dynamic_roles_os.rendered
  filename = "${path.module}/backend_roles.json"
}

But this breaks as the above is looking for role to include an attribute. error

?[31m╷?[0m?[0m
?[31m│?[0m ?[0m?[1m?[31mError: ?[0m?[0m?[1mInvalid reference?[0m
?[31m│?[0m ?[0m
?[31m│?[0m ?[0m?[0m  on dynamic-roles.tf line 266, in data "template_file" "dynamic_roles_os":
?[31m│?[0m ?[0m 266:           "${?[4mrole?[0m}"?[0m
?[31m│?[0m ?[0m
?[31m│?[0m ?[0mA reference to a resource type must be followed by at least one attribute
?[31m│?[0m ?[0maccess, specifying the resource name.
?[31m╵?[0m?[0m
Operation failed: failed running terraform plan (exit 1)

Additionally, I have tried to do the following

locals {

  matching_roles = flatten([
    for key, role_set in data.aws_iam_roles.matching_roles_2 : [
      for arn in role_set.arns : {
        arn = arn
      }
    ]
  ])
}

data "template_file" "dynamic_roles_os" {
  template = <<EOF
{
  "roles": [
    {% for role in roles %}
    {
      "role_name": "all_access",
      "role_mapping_payload": {
        "backend_roles": [
          "${role.arn}"
        ],
        "hosts": [],
        "users": []
      }
    }{% if not loop.last %},{% endif %}
    {% endfor %}
  ]
}
EOF

  vars = {
    roles = local.matching_roles
  }
}
resource "local_file" "backend_roles_json" {
  content  = data.template_file.dynamic_roles_os.rendered
  filename = "${path.module}/backend_roles.json"
}

But this errors with

?[31m│?[0m ?[0m?[1m?[31mError: ?[0m?[0m?[1mReference to undeclared resource?[0m
?[31m│?[0m ?[0m
?[31m│?[0m ?[0m?[0m  on dynamic-roles.tf line 268, in data "template_file" "dynamic_roles_os":
?[31m│?[0m ?[0m 268:           "${?[4mrole.arn?[0m}"?[0m
?[31m│?[0m ?[0m
?[31m│?[0m ?[0mA managed resource "role" "arn" has not been declared in the root module.
?[31m╵?[0m?[0m
Operation failed: failed running terraform plan (exit 1)

So how can I properly create a json template with the ARNs from this data source? I have looked into this Pass list variable to JSON template in terraform But I am still unable to come up with a working solution.

I would expect this to be feasible with the different terraform functions supporting json encoding and template strings.

Solution

I have found a working solution without using the template_file data. Instead using merge(), jsonencode(). This does require having the input file saved as a json file. Would be interested to know if there are other solutions to this problem

locals {
  # Read and decode the input JSON file
  input_json_file  = "${path.module}/input.json"
  output_json_file = "${path.module}/output.json"

  input_json = jsondecode(file(local.input_json_file))

  # Extract ARNs from the matching roles
  flat_arns = flatten([
    for key, role_set in data.aws_iam_roles.matching_roles : role_set.arns
  ])

  # Update the backend_roles in the reserved_roles
  updated_reserved_roles = [
    for role in local.input_json.reserved_roles : merge(role, {
      role_mapping_payload = merge(role.role_mapping_payload, {
        backend_roles = local.flat_arns
      })
    })
  ]

  # Create the modified JSON
  modified_json = merge(local.input_json, {
    reserved_roles = local.updated_reserved_roles
  })
}

resource "local_file" "output_json" {
  content  = jsonencode(local.modified_json)
  filename = local.output_json_file
}

output "modified_json" {
  value = local.modified_json
}

output "output_json_file" {
  value = local_file.output_json.content
}

output "arns" {
  value = local.flat_arns
}