Service account seems unused by Google Cloud Storage in Cloud Function

I am trying to generate signed urls in my python Cloud Functions. I initialize the google-cloud-storage client sdk by :

firebase_admin.initialize_app()
storage_client = storage.Client() # Use default credentials
bucket_name = os.environ.get("BUCKET_NAME")
bucket = storage_client.get_bucket(bucket_name)

And when i deploy my Cloud Function, i specify the service account email using the --service-account flag.

Once it is uploaded and the Function is running the Cloud, i am getting the following error :

you need a private key to sign credentials. the credentials you are currently using <class 'google.auth.compute_engine.credentials.Credentials'> just contains a token. see https://googleapis.dev/python/google-api-core/latest/auth.html#setting-up-a-service-account for more details.

It's working fine if i serve the Cloud Function locally on my Mac using : functions-framework --target myfunction --debug --port=8081

So i am pretty sure this is a service account issue but i am completely lost because there is no other service account i have on my Mac than this one (generated for firebase admin SDK). On my mac i have done :

export GOOGLE_APPLICATION_CREDENTIALS=/Users/foxtom/PycharmProjects/MyProject/firebase-adminsdk.json

And this is the same email inside firebase-adminsdk.json than the one i am passing to gcloud deploy myfunction --service-account="firebase-adminsdk-email@gserviceaccount.com"

Note : which is weird is that the service account is actually used otherwise i could not call the verify_token method from the auth module, because i don't specify a specific account in firebase_admin.initialize_app().

Note 2 : I do see the correct service account in General information under the Cloud Function Details but it's not working only for storage.Client().

What am i missing here ?

EDIT : Additional steps i have made is to add the Storage Object Viewer and Storage Object Creator permissions to this service account from my Cloud Storage bucket but it's still not working (I have also tried using the Storage Admin permission but not working either).

Solution

Do not use a service account key file, especially when you are on Google Cloud. To sign your URL without a service account key file, you can do that

import datetime

from google.cloud import storage

import google.auth.transport.requests
request = google.auth.transport.requests.Request()
creds, _ = google.auth.default()
creds.refresh(request)

def generate_download_signed_url_v4(bucket_name, blob_name):
    """Generates a v4 signed URL for downloading a blob.

    Note that this method requires a service account key file. You can not use
    this if you are using Application Default Credentials from Google Compute
    Engine or from the Google Cloud SDK.
    """
    # bucket_name = 'your-bucket-name'
    # blob_name = 'your-object-name'

    storage_client = storage.Client()
    bucket = storage_client.bucket(bucket_name)
    blob = bucket.blob(blob_name)

    url = blob.generate_signed_url(
        version="v4",
        # This URL is valid for 15 minutes
        expiration=datetime.timedelta(minutes=15),
        # Allow GET requests using this URL.
        method="GET",
        service_account_email="<Service account email>",
        access_token=creds.token

    )

    print("Generated GET signed URL:")
    print(url)
    print("You can use this URL with any user agent, for example:")
    print(f"curl '{url}'")
    return url

if __name__ == '__main__':
    generate_download_signed_url_v4('gib-multiregion-us','access.csv')

The Cloud Function runtime service account must have the role Service Account Token Creator on the service account used to sign the URL (could be itself!)