How to manage password that is used by AWS Kinesis Firehose to write data to a Redshift cluster

Question

I'm trying to write a AWS Cloud Formation template for Kinesis Firehoses that write data to a Redshift Cluster. In RedshiftDestinationConfiguration, The username and password are marked as required.

However, as we will check-in the template to Git we would like to avoid writing password to a file that will be commit to the repo. The target Redshift cluster is not defined in the template, so in the template there will be JDBC URL, username, and password. I believe by those information anyone can see the file can connect to the cluster.

Does anyone know a better way to manage password in this case?

Answer 1

One pattern that you could use is to store your password in a CloudFormation parameter file and encrypt that file using AWS KMS. You would add the unencrypted file to your .gitignore, and add the encrypted binary to your git repo. You'll also need to have added an encryption key to IAM that you can use for this.

The result is that the encrypted file is not readable by anyone that doesn't have IAM permissions to use that encryption key.

Here's an example of how you can use KMS to encrypt and decrypt a file using the AWS CLI.

Encrypt:

aws kms encrypt \
    --key-id xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx \
    --plaintext file://params.json \
    --query CiphertextBlob \
    --output text | base64 --decode > params.json.encrypted

Decrypt:

aws kms decrypt \
    --ciphertext-blob fileb://params.json.encrypted \
    --query Plaintext \
    --output text | base64 --decode > params.json

This comes with the caveat that you need to remember to encrypt a new parameter file and commit that to git, because edits to the unencrypted file won't show up in your git diff. You also need to remember to decrypt when you pull changes from your git repo - you can automate that to make it easier.