How can I insert the correct ec2 instance ID into a dynamic Cloudwatch Dashboard using Terraform?

Question

Background: My organization has multiple AWS accounts that, and within that we have multiple "environments" which we call beta and test. The instances are rebuilt weekly and instance IDs change. So the "Name" tag of a collections of EC2 instances in account 1 will look like this:

profileserver-account1-beta
profileserver-account1-test
domaincontroller-account1-beta
domaincontroller-account1-test

and in account 2 it'll have the same servers, but with "account2" in the name.

MY PROBLEM: I'm trying to build a cloudwatch dashboard with Terraform in our IaC. I want/need the "aws_cloudwatch_dashboard" resource block to iterate over everything and create two dashboards. One for beta, and one for test. I am able to create multiple dashboards, but I can't for the life of me figure out how to get the instance ID's passed properly into each one.

main.tf:

data "aws_instances" "profile_instance" {
  for_each = var.environment

  instance_tags = {
    Name = "profileserver-${var.account}-${each.value}"
  }
}

resource "aws_cloudwatch_dashboard" "cloudwatch_dash" {
  for_each = var.environment
  dashboard_name = "Dashboard-${each.value}"
  dashboard_body = jsonencode({
    widgets: [
        {
            height: 5,
            width: 6,
            y: 0,
            x: 6,
            type: "metric",
            properties: {
                view: "timeSeries",
                metrics: [
                    [ "AWS/ElasticBeanstalk", "RootFilesystemUtil", "EnvironmentName", "profileserver-${var.account}-${each.value}", "InstanceId", "data.aws_instances.profile_instance.ids[0]" ]
                ],
                region: "us-east-1",
                stacked: false,
                title: "Profile Server File System Usage"
            }
        },
  .
  .
  .
}

My problem is two-fold:

1. how do I make sure all beta servers are listed on the beta dashboard, and test servers on the test dashboard?
2. how do I ensure the InstanceId is actually populated? Currently on my dashboard source code the metrics show as the exact code, and NOT the data I want pulled:

            "type": "metric",
            "properties": {
                "metrics": [
                    [ "AWS/EC2", "CPUUtilization", "InstanceId", "data.aws_instances.profileserver-account1-beta.ids[0]" ]

I'm very new and unexperienced in Terraform (or any IAC) but I tried lots of different things to troubleshoot this. I had the JSON in a separate file at first, then I placed under jsonencode. All that did was make writing variables a little easier. I attempted to place a variable inside a variable, but that apparently is not allowed...

Answer 1

Since the data source is using for_each, that means the result will be a set of key value pairs. To fetch the values, a key is necessary. Since both the data source and the resource are using the same variable with for_each, the correct value provided by the data source for a given key can be fetched like this:

data "aws_instances" "profile_instance" {
  for_each = var.environment

  instance_tags = {
    Name = "profileserver-${var.account}-${each.value}"
  }
}

resource "aws_cloudwatch_dashboard" "cloudwatch_dash" {
  for_each = var.environment
  dashboard_name = "Dashboard-${each.value}"
  dashboard_body = jsonencode({
    widgets: [
        {
            height: 5,
            width: 6,
            y: 0,
            x: 6,
            type: "metric",
            properties: {
                view: "timeSeries",
                metrics: [
                    [ "AWS/ElasticBeanstalk", "RootFilesystemUtil", "EnvironmentName", "profileserver-${var.account}-${each.value}", "InstanceId", data.aws_instances.profile_instance[each.key].ids[0] ]
                ],
                region: "us-east-1",
                stacked: false,
                title: "Profile Server File System Usage"
            }
        },
.
.
.
}