I am trying to upload content to a Google bucket using Python SDK in a forked process on a Linux host (Ubuntu 18.04) The operation fails with status code 403:
google.api_core.exceptions.Forbidden: 403 POST https://www.googleapis.com/upload/storage/v1/b/temp-compare/o?uploadType=multipart: ('Request failed with status code', 403, 'Expected one of', )
Example code:
import random
import string
import json
import os
import requests
from datetime import datetime, timedelta
from google.cloud import storage
def upload_to_bucket(plainText):
gcp_client = storage.Client.from_service_account_json("/path/to/google-bucket-credentials.json")
bucket = gcp_client.get_bucket('bucket_name')
bucket_file_name = random_file_name('json', 10)
blob = bucket.blob(bucket_file_name)
blob.upload_from_string(plainText)
url = blob.generate_signed_url(expiration=datetime.utcnow() + timedelta(hours=48), method="GET")
return url
def random_file_name(ext, length):
return "%s.%s" % (''.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(length)), ext)
if __name__ == "__main__":
os.fork()
url = upload_to_bucket("just for test")
res = requests.get(url)
print(res.content.decode('utf8'))
I have tried using the solutions offered in google-api-python-client broken because of OAuth2? with no lock.
How can this be solved?
Not surprisingly it turns out i'm the source of the problem.
After forking the process the random filename created by random_file_name is the same (probably has something to do with the way a seed is generated)
Since the account does not have permission to override the file it gets a 403 response.
If I change random_file_name to be:
def random_file_name(ext, length):
import uuid
return str(uuid.uuid4())[0:length]+"."+ext
All is working as expected.
I would probably be still looking for a solution if it not for @TasosZG advise to use the cloud shell.