How to make Terraform use explicit GCP service account instead of the GCE account it is running on

Question

I'm trying to deploy an application on Google Cloud using Terraform. This deployment should happen during a Gitlab CI job that is running on a Google Compute Engine instance.

I have a service account that should be used, with its location stored in the GOOGLE_APPLICATION_CREDENTIALS environment variable, which Terraform should then use for authentication.

However, as the Terraform docs state, if Terraform is running on a GCE instance, it automatically has access to the instance's service account.

The issue is that Terraform seems to always use the GCE service account and not the service account that I am passing to the environment variable. In fact, even when I explicitly set the credentials property on the provider in my Terraform config (credentials = file("service_account.json")), it still uses the GCE account.

How can I make Terraform use the service account I am giving it instead of the GCE instance account?

I'm using the hashicorp/terraform:light image and version 4.63.0 of the hashicorp/google provider.

Answer 1

By default, Terraform will use the service account of the Google Compute Engine (GCE) instance that it is running on. However, you can explicitly set the GOOGLE_APPLICATION_CREDENTIALS environment variable in your Gitlab CI job to use a different service account. This will override the default behavior of Terraform.

So, first create a environment variable on your Gitlab like GCP_CREDENTIALS_BASE64 and paste the base64-encoded JSON service account content as the value.

After, create a TF variable gcp_credentials_base64 to allow you pass the Gitlab variable to TF :

variable "gcp_credentials_base64" {
  description = "Base64-encoded GCP service account JSON key"
  type        = string
  sensitive   = true
}

Set it on your Gitlab CI job:

deploy:
  image: hashicorp/terraform:light
  script:
    - terraform init
    - terraform apply -auto-approve -var="gcp_credentials_base64=$GCP_CREDENTIALS_BASE64"

Finally, in you main.tf (or wherever the name that you give for your tf file) set the provider use the Terraform that you set in your Gitlab CI, like:

provider "google" {
  credentials = jsondecode(base64decode(var.gcp_credentials_base64))
  project     = "<your-gcp-project-id>"
  region      = "<default-region>"
}

Make sure that the service account JSON file is accessible from the Gitlab CI job and that the path is correct. You may need to adjust the path to match your specific setup.

Note that when using a service account, you also need to grant it the necessary permissions to perform the actions that you want to perform with Terraform. This can be done by assigning the appropriate IAM roles to the service account in the Google Cloud Console.