How do you make Terraform wait for cloud-init to finish?

Question

In my Terraform AWS Docker Swarm module I use cloud-init to initialize the EC2 instance. However, Terraform says the resource is ready before cloud-init finishes. Is there a way of making it wait for cloud-init to finish, ideally without SSHing or checking for a port to be up using a null resource?

Answer 1

Your managers and workers both use template_cloudinit_config. They also have ec2:CreateTags.

You can use an EC2 resource tag like trajano/terraform-docker-swarm-aws/cloudinit-complete to indicate that the cloudinit has finished.

You could add this final part to each to invoke a tagging script:

part { filename = "tag_complete.sh" content = local.tag_complete_script content_type = "text/x-shellscript" }

And declare tag_complete_script be the following:

locals {
  tag_complete_script = <<-EOF
  #!/bin/bash
  instance_id="${TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/instance-id}"
  aws ec2 create-tags --resources "$instance_id" --tags 'Key=trajano/terraform-docker-swarm-aws/cloudinit-complete,Value=true'
  EOF
}

Then with a null_resource, you wait for the tag to appear (wrote this on my phone, so use it for a general idea, but I don't expect that it will work without testing and edits):

resource "null_resource" "wait_for_cloudinit" {
  provisioner "local-exec" {
    command = <<-EOF
    #!/bin/bash
    poll_tags="aws ec2 describe-tags --filters 'Name=resource-id,Values=${join(",", aws_instance.managers[*].id)}' 'Name=key,Values=trajano/terraform-docker-swarm-aws/cloudinit-complete' --output text --query 'Tags[*].Value'"
    expected='${join(",", formatlist("true", aws_instance.managers[*].id))}'
    $tags="$($poll_tags)"
    while [[ "$tags" != "$expected" ]] ; do
      $tags="$($poll_tags)"
    done
    EOF
  }
}

This way you can have dependencies on null_resource.wait_for_cloudinit on any resources that need to run after cloudinit has completed.