Modifying file in-place in Terraform

I have a template file that I'd like to substitute variables in. After substituting and rendering the template file, its file path needs to be used as an input to a module that expects a file.

How do I do that in Terraform? Is there a way I could generate a file in-memory or create and throw away on plan, apply and destroy?

I've tried template, template_dir and templatefile, but all of them only give you access to the file contents as a string, and not as a file handle.

Example

The file templates/config/cluster.xml.tftpl is a Terraform template file that has to have its variables substituted. For the sourceVolume variable, I cannot pass file contents; only the file path.

module "container_definition_1" {
  source  = "cloudposse/ecs-container-definition/aws"
  version = "0.58.2"

  container_name  = "clickhouse-shard1"
  container_image = "clickhouse/clickhouse-server:20.11"
  mount_points = [
    {
      sourceVolume  = "${module.path}/templates/config/cluster.xml.tftpl"
      containerPath = "/etc/clickhouse-server/config.d/cluster.xml"
      readOnly      = false
    }
}

Solution

It's unusual for a Terraform module to take a freshly-generated file as input, and Terraform does not have any built-in primitives for doing so.

However, if there's no way to design this system to avoid a temporary file then you can use the hashicorp/local provider to treat a file on disk as a resource to be managed, with the following caveat mentioned in the documentation:

Note: Terraform primarily deals with remote resources which are able to outlive a single Terraform run, and so local resources can sometimes violate its assumptions. The resources here are best used with care, since depending on local state can make it hard to apply the same Terraform configuration on many different local systems where the local resources may not be universally available. See specific notes in each resource for more information.

Here's one way to write a configuration which I think meets the requirements you gave:

variable "temp_dir" {
  type        = string
  description = "Directory to use for temporary files needed to assemble the container configuration. Must already exist."
}

resource "local_file" "container_xml" {
  filename = "${var.temp_dir}/container.xml"
  content = templatefile("${path.module}/templates/config/cluster.xml.tftpl", {
    # (your template arguments here)
  })
}

module "container_definition_1" {
  source  = "cloudposse/ecs-container-definition/aws"
  version = "0.58.2"

  container_name  = "clickhouse-shard1"
  container_image = "clickhouse/clickhouse-server:20.11"
  mount_points = [
    {
      containerPath = local_file.container_xml.filename
      readOnly      = false
    }
  ]
}

This file will be written to disk rather than only rendered in memory, but you can control where it will be written.

I'd expect that if you use this you'd run Terraform through a wrapper script which generates the temporary directory, passes it to Terraform using terraform apply -var="temp_dir=${filename}" and then cleans up the temporary directory after Terraform has exited. This uses Terraform only as one building-block of your full solution, using the wrapper script to complete the solution because Terraform is not designed to deal with situations like this itself.