Python to get billing info from Gcloud

Question

I am trying to build a small pice of python to get billing info about all projects under organization in google cloud.

I follow the official "how to" https://cloud.google.com/billing/docs/reference/libraries

And after finish all steps (I check that twice) my little program don't works as I spect. I can't get any info or directly I get 403 error.

I think it is problem about perms of the "service account" but this "service account" have the owner perms, as the documentation indicate.

I am currently very lost, to many hours reading and looking for an examples on Internet ... so is why I post here, looking for someone could help me or put in the correct direction.

Let me share with yours my little code in pyhon:

from __future__ import print_function
import os.path
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
from urllib.error import HTTPError
import json

SCOPES = [
    'https://www.googleapis.com/auth/cloud-billing.readonly',
    'https://www.googleapis.com/auth/cloud-billing',
    'https://www.googleapis.com/auth/cloud-platform'
]

JSON_KEY_FILE = '/Users/alek/lab/gcloud-billing/JSON_KEY.json'


def main():
    creds = None

    if os.path.exists(JSON_KEY_FILE):
        creds = ServiceAccountCredentials.from_json_keyfile_name(
            JSON_KEY_FILE, scopes=SCOPES)

    with build('cloudbilling', 'v1', credentials=creds) as service:

        print(service.billingAccounts().list().execute())

        request = service.billingAccounts().list()

        try:
            response = request.execute()
        except HTTPError as e:
            print('Error response status code : {0}, reason : {1}'.format(
                e.status_code, e.error_details))

        print(json.dumps(response, sort_keys=True, indent=4))

        #
        # Second test over a knowing ID
        #

        request = service.billingAccounts().get(
            name="billingAccounts/XXXXXX-YYYYYY-ZZZZZZ")

        try:
            response = request.execute()

        except HTTPError as e:
            print('Error response status code : {0}, reason : {1}'.format(
                e.status_code, e.error_details))




if __name__ == '__main__':
    main()

And the output:

{'billingAccounts': [], 'nextPageToken': ''}
{
    "billingAccounts": [],
    "nextPageToken": ""
}
Traceback (most recent call last):
  File "/Users/alek/lab/gcloud-billing/test01.py", line 73, in <module>
    main()
  File "/Users/alek/lab/gcloud-billing/test01.py", line 47, in main
    response = request.execute()
  File "/Users/alek/.pyenv/versions/gcloud-billing/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/Users/alek/.pyenv/versions/gcloud-billing/lib/python3.9/site-packages/googleapiclient/http.py", line 935, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://cloudbilling.googleapis.com/v1/billingAccounts/XXXXXX-YYYYYY-ZZZZZZ?alt=json returned "The caller does not have permission". Details: "The caller does not have permission">

Answer 1

Your code works for me, but see below...

If you have an Organization (gcloud organizations list returns an organization), see the comments.

If you don't, you need to grant the service account permissions on the billing account:

PROJECT=... # The Project that owns the Service Account
ACCOUNT=... # The Service Account 
EMAIL=${ACCOUNT}@${PROJECT}.iam.gserviceaccount.com

gcloud beta billing accounts add-iam-policy-binding ${BILLING} \
--role=roles/billing.viewer \
--member=serviceAccount:${EMAIL}

Don't forget to enable cloudbilling:

gcloud services enable cloudbilling.googleapis.com \
--project=${PROJECT}

Then it should work!

{'billingAccounts': [{'name': 'billingAccounts/XXXXXX-XXXXXX-XXXXXX', 'open': True, 'displayName': 'personal', 'masterBillingAccount': ''}], 'nextPageToken': ''}
{
    "billingAccounts": [
        {
            "displayName": "billing-account",
            "masterBillingAccount": "",
            "name": "billingAccounts/XXXXXX-XXXXXX-XXXXXX",
            "open": true
        }
    ],
    "nextPageToken": ""
}

Recommendation: use Application Default Credentials

You export GOOGLE_APPLICATION_CREDENTIALS=./${ACCOUNT}.json

And can then:

from googleapiclient.discovery import build

import google.auth


def main():
    creds, project_id = google.auth.default(scopes=SCOPES)
    ...

From scratch:

BILLING=...
PROJECT=...
ACCOUNT=...

EMAIL=${ACCOUNT}@${PROJECT}.iam.gserviceaccount.com

gcloud projects create ${PROJECT}
gcloud beta billing projects link ${PROJECT} \
--billing-account=${BILLING}

gcloud services enable cloudbilling.googleapis.com \
--project=${PROJECT}

gcloud iam service-accounts create ${ACCOUNT} \
--project=${PROJECT}

gcloud iam service-accounts keys create ./${ACCOUNT}.json \
--iam-account=${EMAIL} \
--project=${PROJECT}

gcloud beta billing accounts add-iam-policy-binding ${BILLING} \
--role=roles/billing.viewer \
--member=serviceAccount:${EMAIL}

python3 -m venv venv
source venv/bin/activate

python3 -m pip install google-api-python-client
python3 -m pip install google-auth

export PROJECT
export BILLING
export GOOGLE_APPLICATION_CREDENTIALS=./${ACCOUNT}.json

python3 main.py

When you're done, remove the permission:

gcloud beta billing accounts remove-iam-policy-binding ${BILLING} \
--role=roles/billing.viewer \
--member=serviceAccount:${EMAIL}

NOTE You're using Google's API Client Library for Billing but there's also a Cloud Client Library for Billing. Google Cloud encourages Cloud Client over API Client just be aware of the differences.