Search code examples
terraformamazon-eks

After EKS cluster is created by Terraform, next plan sees subnet changes to tags

I am intending to use Terraform to stand up my entire monitoring infrastructure in AWS. So far in my terraform project have created VPC, subnets, appropriate security groups. I am using the Terraform Registry where possible:

The issue I am seeing is that after the EKS cluster is deployed it introduces tags to the VPC and Subnets that do not appear to be known to Terraform. Hence the next time terraform plan is run it identifying tags that it does not manage and intends to remove them:

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ module.vpc.aws_subnet.private[0]
      tags.%:                                "4" => "3"
      tags.kubernetes.io/cluster/monitoring: "shared" => ""

  ~ module.vpc.aws_subnet.private[1]
      tags.%:                                "4" => "3"
      tags.kubernetes.io/cluster/monitoring: "shared" => ""

  ~ module.vpc.aws_vpc.this
      tags.%:                                "4" => "3"
      tags.kubernetes.io/cluster/monitoring: "shared" => ""


Plan: 0 to add, 3 to change, 0 to destroy.

------------------------------------------------------------------------

There is an issue open with terraform-provider-aws with a local workaround using bash, but does anyone know how to get Terraform to become aware of these tags or to get them to be ignored by subsequent plans in a robust way?

Solution

  • Update

    Terraform now supports a provider wide ignore_tags configuration block.

    This means that one can set up a pattern of tags to ignore from the life cycle.

    E.g.

    provider "aws" {
  ignore_tags {
  key_prefixes = ["kubernetes.io"]
  }
}

    and this will effectively act like a lifecycle ignore config for each resource managed.

    Original answer

    So in the end we chose not to use terraform to deploy the cluster at all, instead we use eksctl the community based tool from Weaveworks.

    https://eksctl.io/

    It was recommended by an AWS solutions architect when we were at the AWS offices in London for some training.

    The config can be stored in source control if needed.

    eksctl create cluster -f cluster.yaml

    Since EKS does a lot of tagging of infrastructure, our lives are much better now the state file is not complaining about tags.