Search code examples
pythonaws-lambdapygsheets

Library errors when running python script on Amazon Linux instance

I am trying to get a script up and running on AWS Lambda and I have had some issues running the script.

To go through each of my problems. I first prepared the script using virtualenv and uploaded it. When I tried to invoke the script, I received the following error.

Payload:
{"errorMessage": "Unable to import module 'handler': No module named 'pygsheets'", "errorType": "Runtime.ImportModuleError"}

After doing that, I found someone who mentioned that they had trouble if they tried to prepare libraries on their system but were able to successfully create the script on an EC2 instance.

After trying that, I was still unsuccessful. As a precautionary measure, I attempted to run the script on the EC2 instance. After adding the location of the library folder, I was able to progress a little farther.

Now I am seemingly caught in an library error loop with the following two errors.

Traceback (most recent call last):
  File "handler.py", line 9, in <module>
    import pygsheets as py
  File "/home/ec2-user/package/pygsheets/__init__.py", line 14, in <module>
    from pygsheets.authorization import authorize
  File "/home/ec2-user/package/pygsheets/authorization.py", line 11, in <module>
    from pygsheets.client import Client
  File "/home/ec2-user/package/pygsheets/client.py", line 8, in <module>
    from pygsheets.drive import DriveAPIWrapper
  File "/home/ec2-user/package/pygsheets/drive.py", line 6, in <module>
    from googleapiclient import discovery
  File "/home/ec2-user/package/googleapiclient/discovery.py", line 50, in <module>
    import google.api_core.client_options
ModuleNotFoundError: No module named 'google.api_core'

and

Traceback (most recent call last):
  File "handler.py", line 9, in <module>
    import pygsheets as py
  File "/home/ec2-user/package/pygsheets/__init__.py", line 14, in <module>
    from pygsheets.authorization import authorize
  File "/home/ec2-user/package/pygsheets/authorization.py", line 6, in <module>
    from google.oauth2 import service_account
ModuleNotFoundError: No module named 'google.oauth2'

If I run python3 -m pip install --target /home/ec2-user/package --upgrade google-auth I get the first error when I run the script. If I then run python3 -m pip install --target ./package --upgrade google-api_core I'll get the second error when I run the script. If I then try to fix the second error I'll just get caught with the first error again.

Does anyone have any idea as to how I can proceed?

Some possibly useful information When I made the virtualenv I did not have any trouble running the script. I only had trouble after uploading to AWS.

To summarize

  • Tried using virtualenv to prepare libraries.
  • No issues when running script in virtualenv locally, but No module named 'pygsheets' error after uploading the files to AWS Lambda.
  • Tried using EC2 instance to prepare. Got No module named 'google.oauth2' error.
  • When trying to troubleshoot on EC2 instance, got a No module loop
Solution

  • I was able to fix my issue. It seems like there was a problem with my Docker. I no longer had any library problems after I reinstalled that software.