Create partition on bigquery table using terraform

Question

Description

I have list of bigquery tables to be created using terraform but I need only the partition for specific tables.

Here is the ex.

locals {
    path = "../../../../../../../../db"

    gcp_bq_tables = [
        "my_table1",
        "my_table1_daily",
        "my_table2",
        "my_table2_daily"
    ]
}

And, the terraform script to create the tables:

resource "google_bigquery_table" "gcp_bq_tables" {
    for_each    = toset(local.gcp_bq_tables)
    dataset_id  = google_bigquery_dataset.gcp_bq_db.dataset_id
    table_id    = each.value
    schema      = file("${local.path}/schema/${each.value}.json")
    labels = {
        env = var.env
        app = var.app
    }
}

In that I need to create partition on timestamp, type as DAY but the columns are different. Lets say for my_table1,

1. The partition column would be my_ts_column_table1 for table1
2. The partition column would be my_last_modified_column_table2 for table2

How to write the terraform script in this scenario.

My exploration

I find a way to do it in terraform_documentation but not sure for multiple tables and how can be specified the partition columns for both tables.

Answer 1

In this case it might be the best to use dynamic [1] with for_each meta-argument [2] to achieve what you want. The code would have to be changed to:

resource "google_bigquery_table" "gcp_bq_tables" {
    for_each    = toset(local.gcp_bq_tables)
    dataset_id  = google_bigquery_dataset.gcp_bq_db.dataset_id
    table_id    = each.value
    schema      = file("${local.path}/schema/${each.value}.json")

    dynamic "time_partitioning" {
      for_each = each.value == "table1" || each.value == "table2" ? [1] : []
      content {
        type  = "DAY"
        field = each.value == "table1" ? "my_ts_column_table1" : "my_last_modified_column_table2"
      }
    }

    labels = {
        env = var.env
        app = var.app
    }
}

---

[1] https://developer.hashicorp.com/terraform/language/expressions/dynamic-blocks

[2] https://developer.hashicorp.com/terraform/language/meta-arguments/for_each