Search code examples
foreachterraformaws-application-load-balancer

Create multiple target groups and listeners using for each


I am trying to create multiple application target groups & listeners in Terraform using For Each. I have somewhat of a complex setup dealing with listener rules to route traffic to target groups based off HTTP headers. Below are the resources I wrote:

resource "aws_lb_listener" "app_listener_forward" {
  for_each          = var.listeners
  load_balancer_arn = aws_lb.app_alb.arn
  port              = each.value.listeners
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-TLS-1-2-Ext-2018-06"
  certificate_arn   = var.ssl_cert

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.app_tg[each.key].arn
  }
}

resource "aws_lb_listener_rule" "app_https_listener_rule" {
  for_each     = var.listeners
  listener_arn = aws_lb_listener.app_listener_forward[each.key].arn

  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.app_tg[each.key].arn
  }

  condition {
    path_pattern {
      values = each.value.paths
    }
  }
}

resource "aws_lb_target_group" "app_tg" {
  for_each    = var.listeners
  name        = each.key
  port        = each.value.app_port
  protocol    = "HTTP"
  target_type = "ip"
  vpc_id      = aws_vpc.app_vpc.id

  health_check {
    interval            = 130
    timeout             = 120
    healthy_threshold   = 10
    unhealthy_threshold = 10
  }

  stickiness {
    type            = "lb_cookie"
    cookie_duration = 86400
  }
}

Below is the variable declaration:

variable "listeners" {
  type = map(object({
    app_port  = number
    paths     = set(string)
    listeners = set(number)
  }))
  default = {
    "app_one" = {
       app_port = 3000
       paths = [
         "/appOne",
         "/appOne/*"
       ]
       listeners = [
         80, 443, 22, 7999, 8999
       ]
    }
    "app_two" = {
       app_port = 4000
       paths = [
         "/appTwo",
         "/appTwo/*"
       ]
       listeners = [
         80, 443, 22, 7999, 8999
       ]
    }
  }
}

Upon trying to execute, I am getting an error dealing with the port attribute of the aws_lb_listener resource. Below is the error:

Error: Incorrect attribute value type
│
│   on alb.tf line 38, in resource "aws_lb_listener" "app_listener_forward":
│   38:   port              = each.value.listeners
│     ├────────────────
│     │ each.value.listeners is set of string with 5 elements
│
│ Inappropriate value for attribute "port": number required.

I tried setting the listeners attribute of the variable to a set (number) and a set (string) due to the list of numbers, but I'm still getting this error.

Any ideas on how to fix this error would be helpful.

Thanks!


Solution

  • Error message state that the value for port in resource "aws_lb_listener" "app_listener_forward" {..} is incorrect as it seems to be.

    As you are already looping for the whole resource using for_each it can not loop inside the values available in the variable used for looping.

    One solution would be to separate the variable listeners into two.

    Step 1: Separate the Variable into 2 variables

    Feel free to use any names which make more sense to you.

    variable "listner_ports" {
      type        = list(string)
      description = "(optional) listeners port numbers"
      default = [
        80, 443, 22, 7999, 8999
      ]
    }
    variable "listeners" {
      type = map(object({
        app_port = number
        paths    = set(string)
      }))
      default = {
        "app-one" = {
          app_port = 3000
          paths = [
            "/appOne",
            "/appOne/*"
          ]
        }
        "app-two" = {
          app_port = 4000
          paths = [
            "/appTwo",
            "/appTwo/*"
          ]
        }
      }
    }
    

    NOTE: I have changed app_{one,two} to app-{one,two} because only alphanumeric characters and hyphens allowed in aws_lb_target_group "name"

    Step 2: Use the terraform dynamic block in aws_lb_listener to use looping for different variables in same resource.

    resource "aws_lb_listener" "app_listener_forward" {
      for_each          = toset(var.listner_ports)
      load_balancer_arn = data.aws_lb.test.arn ## Use aws_lb.app_alb.arn as per your usecase
      port              = each.value
      protocol          = "HTTP" # you might need to use a more complex variable to support ports and protocols but it gives an idea.
    ## Uncommend in your case.
      # ssl_policy        = "ELBSecurityPolicy-TLS-1-2-Ext-2018-06"
      # certificate_arn   = var.ssl_cert
    
      dynamic "default_action" {
        for_each = var.listeners
    
        content {
          type             = "forward"
          target_group_arn = aws_lb_target_group.app_tg[default_action.key].arn
        }
      }
    }
    
    resource "aws_lb_target_group" "app_tg" {
      for_each    = var.listeners
      name        = each.key
      port        = each.value.app_port
      protocol    = "HTTP"
      target_type = "ip"
      vpc_id      = local.vpc_id
    
      health_check {
        interval            = 130
        timeout             = 120
        healthy_threshold   = 10
        unhealthy_threshold = 10
      }
    
      stickiness {
        type            = "lb_cookie"
        cookie_duration = 86400
      }
    }
    

    Just to show you how it works below is the attached plan with only aws_lb_listener and aws_lb_target_group resources.

    Terraform will perform the following actions:
    
      # aws_lb_listener.app_listener_forward["22"] will be created
      + resource "aws_lb_listener" "app_listener_forward" {
          + arn               = (known after apply)
          + id                = (known after apply)
          + load_balancer_arn = "arn:aws:elasticloadbalancing:eu-central-1:xxxxxxxxxxxxxx:loadbalancer/app/stackoverflow/be9c11ed9c543788"
          + port              = 22
          + protocol          = "HTTP"
          + ssl_policy        = (known after apply)
          + tags_all          = (known after apply)
    
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
        }
    
      # aws_lb_listener.app_listener_forward["443"] will be created
      + resource "aws_lb_listener" "app_listener_forward" {
          + arn               = (known after apply)
          + id                = (known after apply)
          + load_balancer_arn = "arn:aws:elasticloadbalancing:eu-central-1:xxxxxxxxxxxxxx:loadbalancer/app/stackoverflow/be9c11ed9c543788"
          + port              = 443
          + protocol          = "HTTP"
          + ssl_policy        = (known after apply)
          + tags_all          = (known after apply)
    
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
        }
    
      # aws_lb_listener.app_listener_forward["7999"] will be created
      + resource "aws_lb_listener" "app_listener_forward" {
          + arn               = (known after apply)
          + id                = (known after apply)
          + load_balancer_arn = "arn:aws:elasticloadbalancing:eu-central-1:xxxxxxxxxxxxxx:loadbalancer/app/stackoverflow/be9c11ed9c543788"
          + port              = 7999
          + protocol          = "HTTP"
          + ssl_policy        = (known after apply)
          + tags_all          = (known after apply)
    
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
        }
    
      # aws_lb_listener.app_listener_forward["80"] will be created
      + resource "aws_lb_listener" "app_listener_forward" {
          + arn               = (known after apply)
          + id                = (known after apply)
          + load_balancer_arn = "arn:aws:elasticloadbalancing:eu-central-1:xxxxxxxxxxxxxx:loadbalancer/app/stackoverflow/be9c11ed9c543788"
          + port              = 80
          + protocol          = "HTTP"
          + ssl_policy        = (known after apply)
          + tags_all          = (known after apply)
    
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
        }
    
      # aws_lb_listener.app_listener_forward["8999"] will be created
      + resource "aws_lb_listener" "app_listener_forward" {
          + arn               = (known after apply)
          + id                = (known after apply)
          + load_balancer_arn = "arn:aws:elasticloadbalancing:eu-central-1:xxxxxxxxxxxxxx:loadbalancer/app/stackoverflow/be9c11ed9c543788"
          + port              = 8999
          + protocol          = "HTTP"
          + ssl_policy        = (known after apply)
          + tags_all          = (known after apply)
    
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
          + default_action {
              + order            = (known after apply)
              + target_group_arn = (known after apply)
              + type             = "forward"
            }
        }
    
      # aws_lb_target_group.app_tg["app-one"] will be created
      + resource "aws_lb_target_group" "app_tg" {
          + arn                                = (known after apply)
          + arn_suffix                         = (known after apply)
          + connection_termination             = false
          + deregistration_delay               = "300"
          + id                                 = (known after apply)
          + ip_address_type                    = (known after apply)
          + lambda_multi_value_headers_enabled = false
          + load_balancing_algorithm_type      = (known after apply)
          + name                               = "app-one"
          + port                               = 3000
          + preserve_client_ip                 = (known after apply)
          + protocol                           = "HTTP"
          + protocol_version                   = (known after apply)
          + proxy_protocol_v2                  = false
          + slow_start                         = 0
          + tags_all                           = (known after apply)
          + target_type                        = "ip"
          + vpc_id                             = "vpc-063017c6abb96eab6"
    
          + health_check {
              + enabled             = true
              + healthy_threshold   = 10
              + interval            = 130
              + matcher             = (known after apply)
              + path                = (known after apply)
              + port                = "traffic-port"
              + protocol            = "HTTP"
              + timeout             = 120
              + unhealthy_threshold = 10
            }
    
          + stickiness {
              + cookie_duration = 86400
              + enabled         = true
              + type            = "lb_cookie"
            }
    
          + target_failover {
              + on_deregistration = (known after apply)
              + on_unhealthy      = (known after apply)
            }
        }
    
      # aws_lb_target_group.app_tg["app-two"] will be created
      + resource "aws_lb_target_group" "app_tg" {
          + arn                                = (known after apply)
          + arn_suffix                         = (known after apply)
          + connection_termination             = false
          + deregistration_delay               = "300"
          + id                                 = (known after apply)
          + ip_address_type                    = (known after apply)
          + lambda_multi_value_headers_enabled = false
          + load_balancing_algorithm_type      = (known after apply)
          + name                               = "app-two"
          + port                               = 4000
          + preserve_client_ip                 = (known after apply)
          + protocol                           = "HTTP"
          + protocol_version                   = (known after apply)
          + proxy_protocol_v2                  = false
          + slow_start                         = 0
          + tags_all                           = (known after apply)
          + target_type                        = "ip"
          + vpc_id                             = "vpc-063017c6abb96eab6"
    
          + health_check {
              + enabled             = true
              + healthy_threshold   = 10
              + interval            = 130
              + matcher             = (known after apply)
              + path                = (known after apply)
              + port                = "traffic-port"
              + protocol            = "HTTP"
              + timeout             = 120
              + unhealthy_threshold = 10
            }
    
          + stickiness {
              + cookie_duration = 86400
              + enabled         = true
              + type            = "lb_cookie"
            }
    
          + target_failover {
              + on_deregistration = (known after apply)
              + on_unhealthy      = (known after apply)
            }
        }
    
    Plan: 7 to add, 0 to change, 0 to destroy.
    

    Hope it helps. and, Yes I have masked the account ID :).

    EDITED

    This section was added later after reading the comment from @David where he mentioned another issue ERROR Message : "InvalidLoadBalancerAction: You cannot specify multiple of the following action type: 'forward'" when trying to create the listener.

    Use this as base code for your aws_lb_listener and I strongly encourage you to modify as per your best possible solution.[Nested Dynamic Blocks --> for documentation refer here ]

    
    resource "aws_lb_listener" "app_listener_forward" {
      for_each          = toset(var.listner_ports)
      load_balancer_arn = aws_lb.test.arn
      port              = each.value
      protocol          = "HTTP" # you might need to use a more complex variable to support ports and protocols but it gives an idea.
    ## Uncommend in your case.
      # ssl_policy        = "ELBSecurityPolicy-TLS-1-2-Ext-2018-06"
      # certificate_arn   = var.ssl_cert
    
      default_action {
        type = "forward"
        forward {
          dynamic "target_group" {
            for_each = var.listeners
            content {
              arn = aws_lb_target_group.app_tg[target_group.key].arn
            }
          }
          stickiness {
            enabled  = true
            duration = 86400
          }
        }
      }
    }
    

    Conclusion: Apply did work with the above code as expected (multiple listeners were created for both target groups with different ports) but can not confirm if that's what you require, however, the solution can be adjusted as per the requirement.