I want to create multiple aws security group rules by reading multiple yaml files located under a different directory using terraform but I am not sure how exactly can I achieve it.
Below is the folder structure:
-> env
|
-> prod
|
-> external
|
-> IXCC
|
-> values.yaml
|
-> IXCB
|
-> values.yaml
my aws_security_group_rules.tf file look like below:
module "security_group_rules" {
source = "../kem-infra-modules/iam-security-group-rules"
for_each = fileset("${path.module}/env/prod/external/*/", "values.yaml")
# Assuming you have output variables from your security_group module
security_group_id = module.security_group.security_group_id
# Use the contents of the YAML file to populate the security group rules
from_port = yamldecode(file("${path.module}/env/prod/external/${dirname(each.value)}/values.yaml"))["fromPort"]
description = yamldecode(file("${path.module}/env/prod/external/${dirname(each.value)}/values.yaml"))["definition"]
protocol = upper(yamldecode(file("${path.module}/env/prod/external/${dirname(each.value)}/values.yaml"))["protocol"])
to_port = yamldecode(file("${path.module}/env/prod/external/${dirname(each.value)}/values.yaml"))["toPort"]
type = yamldecode(file("${path.module}/env/prod/external/${dirname(each.value)}/values.yaml"))["type"]
cidr_blocks = yamldecode(file("${path.module}/env/prod/external/${dirname(each.value)}/values.yaml"))["sourceIpCidrs"]
}
My values.yaml files under each folder looks like below:
IXCB - values.yaml:
fromPort: 443
definition: testing
protocol: tcp
toPort: 443
type: ingress
sourceIpCidrs: ["xx.xx.xx.xx/32", "xx.xx.xx.xx/32", "xx.xx.xx.xx/32"]
IXCC - values.yaml:
fromPort: 443
definition: testing
protocol: tcp
toPort: 443
type: ingress
sourceIpCidrs: ["xx.xx.xx.xx/32", "xx.xx.xx.xx/32", "xx.xx.xx.xx/32"]
I am expecting two security group rules to be created but the plan stage shows 0 to add,0 to change & 0 to delete.
I also tried using locals but that too didnt work:
locals.tf:
locals {
security_group_rules = flatten([
for dir in fileset("${path.module}/env/prod/external", "*") : [
for mapping in yamldecode(file("${path.module}/env/prod/external/${dir}/values.yaml"))["principalMappings"] : {
customer_name = dir
from_port = mapping.fromPort
description = mapping.definition
protocol = upper(mapping.protocol)
to_port = mapping.toPort
type = mapping.type
cidr_blocks = mapping.sourceIpCidrs
}
]
])
}
tried calling this locals into the aws security group rules. it still failed.
Since I don't have the code from the module you are using, I will give an example of how to create security group rules locally:
locals {
sg_rules_path = "${path.module}/env/prod/external"
}
resource "aws_security_group" "test" {
}
resource "aws_security_group_rule" "test" {
for_each = fileset(local.sg_rules_path, "*/values.yaml")
security_group_id = aws_security_group.test.id
description = yamldecode(file("${local.sg_rules_path}/${each.key}")).definition
from_port = yamldecode(file("${local.sg_rules_path}/${each.key}")).fromPort
protocol = upper(yamldecode(file("${local.sg_rules_path}/${each.key}")).protocol)
to_port = yamldecode(file("${local.sg_rules_path}/${each.key}")).toPort
type = yamldecode(file("${local.sg_rules_path}/${each.key}")).type
cidr_blocks = yamldecode(file("${local.sg_rules_path}/${each.key}")).sourceIpCidrs
}
I have named your example YML files sg1 and sg2 and stored them at the same level as the root module. This will yield the following output:
Terraform will perform the following actions:
# aws_security_group.test will be created
+ resource "aws_security_group" "test" {
+ arn = (known after apply)
+ description = "Managed by Terraform"
+ egress = (known after apply)
+ id = (known after apply)
+ ingress = (known after apply)
+ name = (known after apply)
+ name_prefix = (known after apply)
+ owner_id = (known after apply)
+ revoke_rules_on_delete = false
+ tags_all = (known after apply)
+ vpc_id = (known after apply)
}
# aws_security_group_rule.test["sg1/values.yaml"] will be created
+ resource "aws_security_group_rule" "test" {
+ cidr_blocks = [
+ "1.1.1.1/32",
+ "2.2.2.2/32",
+ "3.3.3.3/32",
]
+ description = "testing"
+ from_port = 443
+ id = (known after apply)
+ protocol = "tcp"
+ security_group_id = (known after apply)
+ security_group_rule_id = (known after apply)
+ self = false
+ source_security_group_id = (known after apply)
+ to_port = 443
+ type = "ingress"
}
# aws_security_group_rule.test["sg2/values.yaml"] will be created
+ resource "aws_security_group_rule" "test" {
+ cidr_blocks = [
+ "4.4.4.4/32",
+ "5.5.5.5/32",
+ "6.6.6.6/32",
]
+ description = "testing"
+ from_port = 443
+ id = (known after apply)
+ protocol = "tcp"
+ security_group_id = (known after apply)
+ security_group_rule_id = (known after apply)
+ self = false
+ source_security_group_id = (known after apply)
+ to_port = 443
+ type = "ingress"
}
Having this in mind, I would argue something like the following should work:
locals {
sg_rules_path = "${path.module}/env/prod/external"
}
module "security_group_rules" {
source = "../kem-infra-modules/iam-security-group-rules"
for_each = fileset(local.sg_rules_path, "*/values.yaml")
# Assuming you have output variables from your security_group module
security_group_id = module.security_group.security_group_id
# Use the contents of the YAML file to populate the security group rules
from_port = yamldecode(file("${local.sg_rules_path}/${each.key}")).fromPort
description = yamldecode(file("${local.sg_rules_path}/${each.key}")).definition
protocol = upper(yamldecode(file("${local.sg_rules_path}/${each.key}")).protocol)
to_port = yamldecode(file("${local.sg_rules_path}/${each.key}")).toPort
type = yamldecode(file("${local.sg_rules_path}/${each.key}")).type
cidr_blocks = yamldecode(file("${local.sg_rules_path}/${each.key}")).sourceIpCidrs
}