How to authorize Google Drive API from within an AWS Lambda function?

Question

I am writing a lambda function (for integration with API gateway) that needs to connect to Google Drive. I had working code running locally on my computer using googleapiclient library. I have a credentials.json and client_secret.json file that I am using with OAuth2 to connect to Google Drive. However, when I uploaded these files to my Lambda function, my code is getting an error that it is a read-only file system and the code is trying to write to credentials.json. How can I get this to work? Is there a way I can store and access these files remotely, such as on S3? Or a way to have the authentication code not need to modify credentials.json? This is the code I am using for Google Drive authorization:

from googleapiclient import discovery, errors
from googleapiclient.http import MediaIoBaseDownload, MediaFileUpload
from httplib2 import Http
from oauth2client import client, file, tools

credentials_file_path = './credentials.json'
clientsecret_file_path = './client_secret.json'

SCOPE = 'https://www.googleapis.com/auth/drive'

store = file.Storage(credentials_file_path)
credentials = store.get()

if not credentials or credentials.invalid:
    flow = client.flow_from_clientsecrets(clientsecret_file_path, SCOPE)
    credentials = tools.run_flow(flow, store)
http = credentials.authorize(Http())
drive = discovery.build('drive', 'v3', http=http)

Answer 1

There are two things in your requirements -

1. To read the initial credentials files and
2. To write new data to a file if credentials are invalid or not found.

For the first part which is reading the file you can keep the file in the same root directory as the lambda function python file (which you are already doing).

But for the second part, which is writing to a file it should be done in a file which is located inside /tmp folder because lambda runtime environment doesn't allow you to write to the root directory. More details from AWS documentation - https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtime-environment.html

So your code should write to a file like '/tmp/credentials.json'.

Note: You can also use an S3 bucket to read and write the file.

Another better way to do this is -

Instead of storing your credentials in the file storage of lambda runtime environment, use a service like AWS Secrets Manager to store and retrieve secret files. Details - https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html

This would be much more secure and considered a best practice.