Search code examples
pythongoogle-cloud-platformgoogle-cloud-functionsgoogle-ad-manager

Referencing a YAML file in a GCP bucket from a Google Cloud function where the YAML file also references a json file

I have a Google Cloud function that connects to Google Ad Manager's python API using the below code snippet:

from googleads import ad_manager

client = ad_manager.AdManagerClient.LoadFromStorage('googleads.yaml')

Right now, I have the googleads.yaml as part of the cloud function code (i.e., it gets uploaded as part of the cloud function's source code). I want to move the googleads.yaml to a Google Cloud Storage bucket and then have the Cloud Function read the yaml content by connecting to the google storage bucket.

I successfully did that by doing the below:

def load_credentials_from_gcs(bucket_name, file_path):
    storage_client = storage.Client()
    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.get_blob(file_path)

    if blob is None:
        raise FileNotFoundError(f"The file '{file_path}' does not exist in the bucket '{bucket_name}'.")

    return blob.download_as_text()

def main():
    credentials = load_credentials_from_gcs(bucket_name, file_path)
    client = ad_manager.AdManagerClient.LoadFromString(credentials)

The problem is that the yaml file itself references a json file (json file that contains the service account details). No matter what file path I put for the json file in the yaml, the cloud function always returns the error that the path to the json file is invalid/does not exist. I tried the below options:

  • path_to_private_key_file: "service-account-creds.json" ; assuming that since both yaml and json files are in the bucket, that the yaml file would be able to read the file with this path.
  • path_to_private_key_file: "gs://bucket-name/service-account-creds.json"
  • path_to_private_key_file: "https://storage.cloud.google.com/bucket-name/service-account-creds.json"

Can anyone help me fix this issue?

Also, if putting the yaml and json credentials files in a Google Cloud Storage bucket is not the best way to protect these files. What is another good approach?

Solution

  • As per your object’s absolute path :

    gs://bucket-name/service-account-creds.json

    Your parameters should be :

    bucket = bucket-name
object_name = service-account-creds.json

    And the way to access it as follows :

    from google.cloud import storage

bucket_name = "bucket-name"
object_name = "service-account-creds.json"

client = storage.Client()
bucket = client.bucket(bucket_name)
blob = bucket.blob(object_name)

# download to memory
contents = blob.download_as_bytes()

# download to a local file (with same folder structure as blob storage)
blob.download_to_filename(object_name)

    You were using download_as_text which will retrieve just text as a result, but you will need the json file as a credential so you have to use the download_to_filename which will give you the json file.

    Also, putting the yaml and json credentials files in a Google Cloud Storage bucket is not the best way to protect these files. What is another good approach?

    Storing the YAML and JSON credentials files in a Cloud Storage bucket is not the best way to protect these files. As anyone who has access to that bucket can read and edit the objects inside it. You can set appropriate bucket access control permissions to limit who can read the credentials from the bucket. Use the principle of least privilege and grant access only to the service account used by the Cloud Function.

    OR

    Instead you can use Cloud Key Management Service (KMS) API

    OR

    Use cloud functions environmental variables as Instead of storing the credentials in a separate file, you can store them as environment variables in the Cloud Function configuration

    Reference :