Search code examples
hclterraform-template-filenomadconsul-template

Nomad job: interpolating within Docker args


I'm writing a Thanos sidecar job for Prometheus. I have looked and can't figure out how to specify this argument in a config file, so my job looks like this. This is a templatefile for Terraform, so if you try to reference ${var}, Terraform tries to evaluate that as a templatefile variable.

  task "thanos-sidecar" {
      driver = "docker"

      config {
        image = "quay.io/thanos/thanos:v0.30.2"
        args = [
          "sidecar",
          "--tsdb.path=/alloc/data/tsdb",
# XXX BUSTED
# variants using single or double curlies break the job. HALP.
#          "--prometheus.url=http://$NOMAD_ADDR_prometheus_ui",  # doesn't evaluate
          "--grpc-address=0.0.0.0:10901",
          "--http-address=0.0.0.0:10902",
          "--objstore.config-file=/local/thanos.yml"
        ]
      } # end config

      template {
        change_mode = "restart"
        destination = "local/thanos.yml"

        data = <<EOH
#{{ env "NOMAD_ADDR_prometheus_ui" }} # This evaluates as expected!
type: S3
config:
  bucket: metrics
{{ with service "olly-${olly_name}.ssnc-olly-minio-s3" }}
{{ with index . 0 }}
  endpoint: {{ .Address }}:{{ .Port }} # {{ .Node }}
{{ end }}{{ end }}
  signature_version2: false
  access_key: minioadmin
  secret_key: minioadmin
EOH
      } # end template


      resources {
        memory = 1024
      } # end resources
    } # end task

I've tried a number of different syntax options. I'm almost at the point of using Terraform to ask Consul where the job is and pass in the address to be evaluated in the templatefile. That's less than ideal for a few reasons, not least of which is that the job might get rescheduled but Terraform might not know that, so the address could change without being reformulated in the config file.

I looked also at using template sequences, but no dice.


Solution

  • And...the solution is easy enough. Use the format $${ NOMAD_ADDR_prometheus_ui } - note the double dollar sign, so that Terraform's templatefile() won't interpret it at the first stage.