Search code examples
pythondockergoogle-cloud-platformgoogle-secret-manager

Accessing secrets in GCP Secret Manager from Python in Docker- nontsop Permission Denied

I'm building a simple app which stores Twilio credentials in GCP Secret Manager and pulls them down when it needs them. However, I keep receiving denied permissions errors (403) on the project resource:

google.api_core.exceptions.PermissionDenied: 403 Permission denied on resource project .

I am using an environment variable set to the path to a JSON file containing the credentials for a service account.

Here's what I have tried already:

  • Ensuring that the permissions are set correctly in the GCP Console. The service account is set as an Owner for the project and as a Secret Accessor at the project level, AND as a Secret Accessor at the object level for each of the secrets.
  • Ensuring that the environment variable is set correctly- I have verified that the ENV variable is set correctly and that the file to which it points can be read. I can print the contents of the file by opening the ENV variable as a JSON file.
  • Confirmed that the auth info is correct, by comparing contents of my JSON file to the data in the GCP console
  • I have used the gcloud CLI to login under the service account, and then used the CLI commands to retrieve the very same secrets
  • I can successfully access and push data to a GCS bucket, suggesting that the credentials are correctly loaded from the ENV variable
  • I have tried accessing the secrets in many ways. I've tried other methods, such as listing the secrets in the project. All return a permissions error.

As a reference, I've been following the instructions found at https://cloud.google.com/secret-manager/docs/reference/libraries#client-libraries-install-python and have also been using the official client library docs to explore what else could be wrong. Nothing has really helped me out here.

I've read every resource I can find, and nothing is helping. Any thoughts?

Thank you!!!

EDIT: Adding code below:

def access_secret(project_id, secret_id, version):
    """
    Access a secret- API token, etc- stored in Secret Manager

    Code from https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets#secretmanager-access-secret-version-python
    """
    client = secretmanager.SecretManagerServiceClient()

    # Build the resource name of the secret version
    name = client.secret_version_path(project_id, secret_id, version)

    # Access the secret version
    response = client.access_secret_version(name)

    # Return the secret payload
    payload = response.payload.data.decode('UTF-8')

    return payload

EDIT2: Here's the Dockerfile I'm running this code in:

FROM python:3.8.2-slim-buster

WORKDIR /build

# Copy in the requirements.txt file and service account credentials
COPY requirements.txt <CREDENTIALS_FILENAME>.json /build/ 

ENV PYTHONUNBUFFERED=1 \
    GOOGLE_APPLICATION_CREDENTIALS=/build/<CREDENTIALS_FILENAME>.json \
    VOICEMAIL_TIMEOUT=55 \
    MULTIRING_TIMEOUT=15 \
    GCS_VM_BUCKET=<MY GCS BUCKET NAME> \
    GCP_PROJECT=<MY GCP PROJECT NAME> \
    PHONE_NUMBER=<PHONE NUMBER> \
    TWILIO_ACCOUNT_SID_VERSION=1 \
    TWILIO_AUTH_TOKEN_VERSION=1

# Install packages
RUN pip install -r requirements.txt

# Navigate to the directory containing the Python code
WORKDIR /code/src/

EXPOSE 5000

# Run the actual Python code
CMD ["python", "main.py"]

Later, I have Python code that calls the function above:

GCS_VM_BUCKET = os.environ['GCS_VM_BUCKET']
GCP_PROJECT = os.environ['GCP_PROJECT']

TWILIO_SID = access_secret(GCP_PROJECT, 'TWILIO_ACCOUNT_SID', os.environ['TWILIO_ACCOUNT_SID_VERSION'])
TWILIO_AUTH_TOKEN = access_secret(GCS_PROJECT, 'TWILIO_AUTH_TOKEN', os.environ['TWILIO_AUTH_TOKEN_VERSION'])

where TWILIO_ACCOUNT_SID and TWILIO_AUTH_TOKEN are the names of the secrets in GCP.

Full error trace:

Traceback (most recent call last):   

File "/usr/local/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 57, in error_remapped_callable                                                                                                        return callable_(*args, **kwargs)                                                                                                                                                                                     
File "/usr/local/lib/python3.8/site-packages/grpc/_channel.py", line 826, in __call__                                                                                                                                     
return _end_unary_response_blocking(state, call, False, None)                                                                                                                                                         
File "/usr/local/lib/python3.8/site-packages/grpc/_channel.py", line 729, in _end_unary_response_blocking                                                                                                                 
raise _InactiveRpcError(state)                                                                                                                                                                                      
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:                                                                                                                                                
status = StatusCode.PERMISSION_DENIED                                                                                                                                                                                   
details = "Permission denied on resource project <MY GCP PROJECT NAME>."                                                                                                                                                       
debug_error_string = "{"created":"@1588389938.954039708","description":"Error received from peer ipv4:172.217.12.138:443","file":"src/core/lib/surface
/call.cc","file_line":1056,"grpc_message":"Permission denied on resource 
project <MY GCP PROJECT NAME>.","grpc_status":7}"                                                                                                                                                               >                                                                                                                                                                                                                                                                                                                                                                                                                                               
The above exception was the direct cause of the following exception:                                                                                                                                                                                                                                                                                                                                                                            
Traceback (most recent call last):                                                                                                                                                                                        
File "main.py", line 7, in <module>                                                                                                                                                                                       
from parameters import *                                                                                                                                                                                              
File "/code/src/parameters.py", line 16, in <module>                                                                                                                                                                      
TWILIO_SID = access_secret(GCP_PROJECT, 'TWILIO_ACCOUNT_SID', 
os.environ['TWILIO_ACCOUNT_SID_VERSION'])                                                                                                         
File "/code/src/utils.py", line 46, in access_secret                                                                                                                                                                      
response = client.access_secret_version(name)                                                                                                                                                                         
File "/usr/local/lib/python3.8/site-packages/google/cloud/secretmanager_v1
/gapic/secret_manager_service_client.py", line 963, in access_secret_version                                                                    
return self._inner_api_calls["access_secret_version"](                                                                                                                                                                
File "/usr/local/lib/python3.8/site-packages/google/api_core/gapic_v1
/method.py", line 143, in __call__                                                                                                                   
return wrapped_func(*args, **kwargs)                                                                                                                                                                                  
File "/usr/local/lib/python3.8/site-packages/google/api_core/retry.py", line 
281, in retry_wrapped_func                                                                                                                   
return retry_target(                                                                                                                                                                                                  
File "/usr/local/lib/python3.8/site-packages/google/api_core/retry.py", line 
184, in retry_target                                                                                                                         
return target()                                                                                                                                                                                                       
File "/usr/local/lib/python3.8/site-packages/google/api_core/timeout.py", 
line 214, in func_with_timeout                                                                                                                  
return func(*args, **kwargs)                                                                                                                                                                                          
File "/usr/local/lib/python3.8/site-packages/google/api_core
/grpc_helpers.py", line 59, in error_remapped_callable                                                                                                        
six.raise_from(exceptions.from_grpc_error(exc), exc)                                                                                                                                                                  
File "<string>", line 3, in raise_from                                                                                                                                                                                
google.api_core.exceptions.PermissionDenied: 403 Permission denied on 
resource project <MY GCP PROJECT NAME>.
Solution

  • OK, turns out I was using the project NAME and not the project ID.

    sigh. At least the answer was an easy one. :]