Using a database password stored in Google Secrets Manager from within a pod running in a GKE cluster

Question

I am running an application in a Google Kubernetes Cluster. One pod in it needs access to an externally managed database (Google cloud SQL), and up until now has had a hard coded password for testing purposes. It's come time to move to a more secure solution, so a generated password is now in my Google Secrets Manager and has replaced my user password in my database.

What is the most efficient yet secure way for me to allow the pod to access the database?

My deployment process is as follows:

1. Source changes pushed to source repo
2. Changes trigger Cloud Build
3. Cloud Build builds, tags, and pushes images.
4. Cloud Build commits new image version to source repo
5. Changes to Kubernetes base manifest trigger Cloud Build
6. Cloud Build uses Kustomize to generate new Kubernetes manifest and pushes it to my GKE

Answer 1

There are might be some options that avoid having the password pass through some of the systems that you mentioned (specifically Cloud Build and Kustomize).

I think that the most secure way would be using a configuration described here: https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine that way you can avoid the password all together and rely on IAM controls.

If this setup doesn't work for you-- Would using workload identity from your GKE app to access the Secret Manager API directly an option? This would allow your workload to access Secret Manager directly through IAM controls. https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity

If your app can't be modified to make API calls to Secret Manager there is also an CSI driver so that you can use the file system or environment variables: https://github.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp