Search code examples
kubernetesterraformkubeadmbare-metal-servermetallb

Terraform apply, how to increment count and add kubernetes worker nodes to the existing workers?

I deployed k8s cluster on bare metal using terraform following this repository on github

Now I have three nodes:

ewr1-controller, ewr1-worker-0, ewr1-worker-1

Next, I would like to run terraform apply and increment the worker nodes (*ewr1-worker-3, ewr1-worker-4 ... *) while keeping the existing controller and worker nodes. I tried incrementing the count.index to start from 3, however it still overwrites the existing workers.

resource "packet_device" "k8s_workers" {
  project_id       = "${packet_project.kubenet.id}"
  facilities       = "${var.facilities}"
  count            = "${var.worker_count}"
  plan             = "${var.worker_plan}"
  operating_system = "ubuntu_16_04"
  hostname         = "${format("%s-%s-%d", "${var.facilities[0]}", "worker", count.index+3)}"
  billing_cycle    = "hourly"
  tags             = ["kubernetes", "k8s", "worker"]
}

I havent tried this but if I do 

terraform state rm 'packet_device.k8s_workers'

I am assuming these worker nodes will not be managed by the kubernetes master. I don't want to create all the nodes at beginning because the worker nodes that I am adding will have different specs(instance types).

The entire script I used is available here on this github repository. I appreciate it if someone could tell what I am missing here and how to achieve this.

Thanks!

Solution

  • I solved this issue by modifying and removing modules, resources from the terraform.state: manually modifying it and using terraform state rm <--->.

    • Leaving out (remove state) the section that I want to keep as they are.

    • Modifying the sections in terraform.state that I want change when new servers are added.

    • Incrementing the counter to add new resources, see terraform interpolation.

    I am using bare-metal cloud provider to deploy k8s and it doesn't support k8s HA or VA autos-calling. This is may not be optimal solution as other have pointed out but if it is not something you need to do quite often, terraform can do the job albeit the hardway.