Storing REST API credentials safely for access in a Python environment

Question

I have a number of REST APIs from a software program (ex: Tradingview)

I would like to store the API credentials (e.g. keys, secrets) safely.

I had thought about placing them in a Database table - but - I am not totally fond of placing clear text in a table.

I already know about using OS Environment Variables:

[... snip ...]

import os
import sys
import logging

[... snip ...]

LD_API_KEY      = os.getenv("BINANCE_APIKEY")
LD_API_SECRET   = os.getenv("BINANCE_API_SECRET")

where keys are stored in a file - but - as mentioned before, I have a number of API keys.

Just leaving them on a server (in clear text) - even though the file is hidden - is not sitting well with me.

Is there any other way to store API Keys?

Answer 1

There are a number of articles on this topic, a quick web search for "Storing API keys" will net you some interesting and informative reads, so I'll just talk about my experience here.

Really, it's all up to preference, the requirements of your project, and the level of security you need. I personally have run through a few solutions. Here's how my project has evolved over time.

Each key stored in environment variables

Simple enough, just had to use os.environ for every key. This very quickly became a management headache, especially when deploying to multiple environments, or setting up an environment for a new project contributor.

All keys stored in a local file

This started as just a file outside source control with an environment variable pointing to the file. I started with a simple JSON file in the following structure.

[
  {
    "name": "Service Name",
    "date": "1970-01-01",  // to track rotations
    "key": "1234abcd",
    "secret_key": "abcd1234"
  }
]

This evolved into a class that accessed this file for me and returned the desired key so I didn't have to repeat json.load() or import os in every script that accessed APIs. This got a little more complex when I started needing to store OAuth tokens.

I eventually moved this file to a private, encrypted (git-secret), local-only git repo so team members could also use the keys in their environments.

Use a secret management service

The push to remote work forced me to create a system for remote API key access and management. My team debated a number of solutions, but we eventually fell on AWS Secrets Manager. The aforementioned custom class was pointed at AWS instead of a local file, and we gained a significant increase in security and flexibility over the local-only solution.

There are a number of cloud-based secret management solutions, but my project is already heavily AWS integrated, so this made the most sense given the costs and constraints. This also means that each team member now only needs to have AWS permissions and use their accounts AWS API key for access.