Create managed private endpoint for Synapse using terraform

Question

I need to create Managed private endpoint (MPE) for Synapse to Storage Accounts using terraform. I understand we have a resource block to create MPE. However the challenge I have is I may have to read from a YAML like below. It may have multiple storage account's in each workspace to create MPE's. And there may be multiple Synapse instances with storage accounts.

resource_endpoints:
  - source_name: synapse_workspace_1
    source_subscription_id: sub-fedcba9876543210
    source_resource_group: rg-sourceGroup2
    source_type: synapse
    resources:
      - destination_type: Blob Storage
        destination_name: sa2675asj7234asf
        destination_subscription_id: sub-fedcba9876543210
        destination_resource_group: rg-destinationGroup3
      - destination_type: Blob Storage
        destination_name: saasdjkf23q876h23sd
        destination_subscription_id: sub-abcdef1234567890
        destination_resource_group: rg-destinationGroup2
        
  - source_name: synapse_workspace_1
    source_subscription_id: sub-fedcba9876543210
    source_resource_group: rg-sourceGroup2
    source_type: synapse
    resources:
      - destination_type: Blob Storage
        destination_name: sa2675asj7234asf
        destination_subscription_id: sub-fedcba9876543210
        destination_resource_group: rg-destinationGroup3
      - destination_type: Blob Storage
        destination_name: saasdjkf23q876h23sd
        destination_subscription_id: sub-abcdef1234567890
        destination_resource_group: rg-destinationGroup2

The resource block I used is below

resource "azurerm_synapse_managed_private_endpoint" "synapse_private_endpoint" {
  for_each = {for idx, sws local.synworkspace => ??? }
  name                = "mpe-some-randomstring"
  synapse_workspace_id = each.value.synapse_workspace_id
  target_resource_id  = each.value.target_resource_id
  subresource_name    = each.value.subresource_name
}

I know to read YAML in terraform. I'm wondering how I can loop through the storage accounts in this resource block for the YAML file to create MPE's. We can use "data" block to check information on Storage Account. But again the challenge is how do I have a loop resource block and use data block as well. This feels more complicated. Should I think of another way to achieve the same

Answer 1

If I understood correctly, you need to iterate over all the resources in a yaml file - let's call it resources.yaml:

resource_endpoints:
  - source_name: synapse_workspace_1
    source_subscription_id: sub-fedcba9876543210
    source_resource_group: rg-sourceGroup2
    source_type: synapse
    resources:
      - destination_type: Blob Storage
        destination_name: sa2675asj7234asf
        destination_subscription_id: sub-fedcba9876543210
        destination_resource_group: rg-destinationGroup3
      - destination_type: Blob Storage
        destination_name: saasdjkf23q876h23sd
        destination_subscription_id: sub-abcdef1234567890
        destination_resource_group: rg-destinationGroup2

  - source_name: synapse_workspace_1
    source_subscription_id: sub-fedcba9876543210
    source_resource_group: rg-sourceGroup2
    source_type: synapse
    resources:
      - destination_type: Blob Storage
        destination_name: sa2675asj7234asf
        destination_subscription_id: sub-fedcba9876543210
        destination_resource_group: rg-destinationGroup3
      - destination_type: Blob Storage
        destination_name: saasdjkf23q876h23sd
        destination_subscription_id: sub-abcdef1234567890
        destination_resource_group: rg-destinationGroup2

You can use the flatten() function to create an array of resources but which include the source info as well like this:

locals {
  settings = yamldecode(file("${path.module}/resources.yaml"))

  resources = flatten([
    for endpoint in local.settings.resource_endpoints : [
      for resource in endpoint.resources : {
        source_name            = endpoint.source_name
        source_subscription_id = endpoint.source_subscription_id
        source_resource_group  = endpoint.source_resource_group
        source_type            = endpoint.source_type

        destination_type            = resource.destination_type
        destination_name            = resource.destination_name
        destination_subscription_id = resource.destination_subscription_id
        destination_resource_group  = resource.destination_resource_group
      }
    ]
  ])
}

output "resources" {
  value       = local.resources
  description = "Flattened list of resources."
}

Output of terraform plan:

Changes to Outputs:
  + resources = [
      + {
          + destination_name            = "sa2675asj7234asf"
          + destination_resource_group  = "rg-destinationGroup3"
          + destination_subscription_id = "sub-fedcba9876543210"
          + destination_type            = "Blob Storage"
          + source_name                 = "synapse_workspace_1"
          + source_resource_group       = "rg-sourceGroup2"
          + source_subscription_id      = "sub-fedcba9876543210"
          + source_type                 = "synapse"
        },
      + {
          + destination_name            = "saasdjkf23q876h23sd"
          + destination_resource_group  = "rg-destinationGroup2"
          + destination_subscription_id = "sub-abcdef1234567890"
          + destination_type            = "Blob Storage"
          + source_name                 = "synapse_workspace_1"
          + source_resource_group       = "rg-sourceGroup2"
          + source_subscription_id      = "sub-fedcba9876543210"
          + source_type                 = "synapse"
        },
      + {
          + destination_name            = "sa2675asj7234asf"
          + destination_resource_group  = "rg-destinationGroup3"
          + destination_subscription_id = "sub-fedcba9876543210"
          + destination_type            = "Blob Storage"
          + source_name                 = "synapse_workspace_1"
          + source_resource_group       = "rg-sourceGroup2"
          + source_subscription_id      = "sub-fedcba9876543210"
          + source_type                 = "synapse"
        },
      + {
          + destination_name            = "saasdjkf23q876h23sd"
          + destination_resource_group  = "rg-destinationGroup2"
          + destination_subscription_id = "sub-abcdef1234567890"
          + destination_type            = "Blob Storage"
          + source_name                 = "synapse_workspace_1"
          + source_resource_group       = "rg-sourceGroup2"
          + source_subscription_id      = "sub-fedcba9876543210"
          + source_type                 = "synapse"
        },
    ]

Ideally, local.resources should be created as a map of objects. But in your yaml file you have both endpoints and resources duplicated, so I couldn't understand what property (or properties) could be used as a key to the map.