AccessDeniedException: UnknownError when creating a Lambda using Terraform

Question

I am trying to create a simple Lambda through Terraform. I'm getting an error that has popped up on Stack Overflow plenty of times and I'm reading through the solutions and they're not working for me.

When I terraform apply get the this error.

aws_lambda_function.get_snow: Creating...
╷
│ Error: creating Lambda Function (howmuchsnow_get_snow): operation error Lambda: CreateFunction, https response error StatusCode: 403, RequestID: b66dcc48-58f8-4806-9100-69950c011cd4, api error AccessDeniedException: UnknownError
│ 
│   with aws_lambda_function.get_snow,
│   on get-snow-lambda.tf line 58, in resource "aws_lambda_function" "get_snow":
│   58: resource "aws_lambda_function" "get_snow" {
│ 

I have seen many SO posts on this exact error and all of them have been resolved by giving the appropriate permissions to the the IAM user performing the action of terraform apply on the command line. However, my IAM user has administrator access.

This is the IAM user I am connected as in the environment of my terraform session.

My project is two files.

terraform-backend.tf

terraform {
  required_version = ">= 1.2.7"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.19.0"
    }
  }

  backend "s3" {
    bucket         = "howmuchsnow-terraform-state"
    key            = "global/s3/terraform.tfstate"
    region         = "us-east-1" # Can't use a variable here - maybe an env var?
    dynamodb_table = "howmuchsnow-terraform-lock"
    encrypt        = true
  }
}

provider "aws" {
  region = var.aws_region
}

resource "aws_s3_bucket" "terraform_state" {
  bucket = "howmuchsnow-terraform-state"
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_dynamodb_table" "terraform_lock" {
  name         = "howmuchsnow-terraform-lock"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"
  attribute {
    name = "LockID"
    type = "S"
  }
}

get-snow-lambda.tf

data "aws_iam_policy_document" "get_snow" {
  statement {
    effect  = "Allow"
    actions = ["sts:AssumeRole"]
    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "get_snow_lambda" {
  name               = "get_snow_lambda_role"
  assume_role_policy = data.aws_iam_policy_document.get_snow.json
}

resource "aws_cloudwatch_log_group" "get_snow_lambda" {
  name = "/aws/lambda/${aws_lambda_function.get_snow.function_name}"
  retention_in_days = var.log_retention
}

data "aws_iam_policy_document" "get_snow_lambda" {
  statement {
    actions = [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:PutLogEvents",
    ]
    resources = ["arn:aws:logs:*:*:*"]
  }
}

resource "aws_iam_role_policy" "get_snow_lambda" {
  name   = "howmuchsnow_get_snow_lambda_policy"
  policy = data.aws_iam_policy_document.get_snow_lambda.json
  role   = aws_iam_role.get_snow_lambda.id
}

data "archive_file" "get_snow_lambda" {
  type        = "zip"
  source_file = "${path.module}/lambdas/get_snow.py"
  output_path = "${path.module}/lambdas/get_snow.zip"
}

resource "aws_lambda_function" "get_snow" {
  function_name    = "howmuchsnow_get_snow"
  filename         = data.archive_file.get_snow_lambda.output_path
  handler          = "get_snow.handler"
  source_code_hash = data.archive_file.get_snow_lambda.output_base64sha256
  runtime          = "python3.10"
  role             = aws_iam_role.get_snow_lambda.arn
}

Also For reference:

Terraform v1.6.0
on linux_amd64
+ provider registry.terraform.io/hashicorp/archive v2.4.0
+ provider registry.terraform.io/hashicorp/aws v5.19.0

Answer 1

I turns out that this issue was actually with an account closure for suspicious activity. In the console, this is not mentioned in any obvious place. All services could be managed while this was going on and most of them worked. Lambda, was given an "Unknown error" and could not be managed from the command line either. I ended up contacting AWS and they told me this was because they thought they say suspicious activity which was really just things I needed to clean up from when my friends and I did a project that wasn't costing me anything. I think their error reporting needs a lot of work.