Unzip and read a .gz file from Google Cloud storage via Python (Cloud Function)

Question

I have to read an unzip file from google cloud storage in python (cloud function). I tried below method but the CF crashes everytime. File in GCS : ABC.gz

import gzip  

def process(data, context):  
    filename = data['name']  
    with gzip.open("'"+filename+"'", 'rb') as f:  
        file_content = f.read()

Please suggest.

Answer 1

This is an expected behavior. When your function is triggered by a Cloud Storage event you only get the a dictionary containing the data for the event, not the object itself.

You should get the object from Cloud Storage using the name and bucket from the data dictionary. Here's a code that can guide you to get the file which is taken from here:

import gzip
from google.cloud import storage

def process(data, context):
    storage_client = storage.Client()

    bucket = storage_client.bucket(data['bucket'])
    blob = bucket.blob(data['name'])
    blob.download_to_filename("/tmp/" + data['name'])

    #Here goes your code to unzip the file

Consider that you may need to add the needed libraries to the requirements.txt file and grant the needed permissions to the Runtime Service Account of the Function to access Cloud Storage.