How can I continue Terraform's execution if a data source does not find any results?

Question

This command works perfectly fine for me via the terminal:

aws ssm get-parameter --name "/something/i/know/exists" --query "Parameter.Value" --output text

Output:

subnet-whatIwouldExpect

But setting it as a variable inside Terraform has been a different story completely.

(I have been happily using Terraform plan/apply/destroy etc)

data "external" "ssm_parameter_value" {
  program = ["bash", "-c", <<-EOF
    result=$(aws ssm get-parameter --name "/something/i/know/exists" --query "Parameter.Value" --output text)
    echo "{\"result\": \"$result\"}"
  EOF
  ]
}

output "ssm_parameter_result" {
  value = data.external.ssm_parameter_value.result
}

When I try terraform refresh I get:

ssm_parameter_result = tomap({
  "result" = ""
})

This is extremely frustrating, and I don't know what I'm doing wrong.

This is my entire file:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.34.0"
    }
  }
  
  required_version = ">= 0.14.9"
}

provider "aws" {
  region = "eu-west-2"
  default_tags {
    tags = local.common_tags
  }
}

data "aws_caller_identity" "current" {}

data "aws_region" "current" {}

# Execute AWS CLI command to get SSM parameter value
data "external" "ssm_parameter_value" {
  program = ["bash", "-c", <<-EOF
    result=$(aws ssm get-parameter --name "/something/i/know/exists" --query "Parameter.Value" --output text)
    echo "{\"result\": \"$result\"}"
  EOF
  ]
}

output "ssm_parameter_result" {
  value = data.external.ssm_parameter_value.result
}

---

EDIT:

I am aware of data "aws_ssm_parameter" "example" { name = "/A/B/C" }

However, this fails with ParameterNotFound if it doesn't exist on AWS.

And it does not seem as though this is surpsa

I want to be able to utilise the value of

data.external.ssm_parameter_value.result

to drive the creation/non-creation of other resources later on:

e.g.

resource "aws_security_group_rule" "example" {  
  count = data.external.ssm_parameter_value.result != "" ? 1 : 0
  type              = "ingress"
  from_port         = 0
  to_port           = 0
  protocol          = "-1"
  security_group_id = data.external.ssm_parameter_value.result["result"]
  source_security_group_id = aws_security_group.another-sg.id
  
  description = "All Traffic from Lambda"
}

Answer 1

As Mark explained in his answer you should, when possible, use resource specific data sources. Data sources, however, will always error out if the resource does not exist. An enhancement request was made years ago to request a change in this behavior, however it was declined.

So in this case however the external data source is a valid alternative.

The external data source mandates that the program output must be a JSON object. In this example, you could do:

# Execute AWS CLI command to get SSM parameter value
data "external" "ssm_parameter_value" {
  program = ["bash", "-c", <<-EOF
    param=$(aws ssm get-parameter --name "/something/i/know/exists" --query "Parameter.Value" --output text 2>/dev/null);
    if [ "$?" -eq 0 ]; then 
      jq -r -n --arg param "$param" '{"param":$param}'
    else
      jq -r -n '{"param":""}'
    fi
  EOF
  ]
}

output "ssm_parameter_result" {
  value = data.external.ssm_parameter_value.result.param
}

If the secret exists, the output will be:

$ terraform apply
...
ssm_parameter_result = "foo"

If the secret does not exist, the output will be:

$ terraform apply
...
ssm_parameter_result = ""

Note that this approach requires jq for the creation of the JSON object.