`No module named...` while deploying an AWS Lambda using Serverless Framework, poetry, python3.10 and fastapi

Question

EDIT

Il looks like theproblem comes from the Serverless plugin, serverless-python-requirements.

When packaging with $ sls package, when having python = "^3.9" the dependencies are in the zip, but having python = "^3.10" they are not. Everything else is the same.

end of edit

---

I want to deploy an AWS Lambda using Serverless Framework, poetry, python3.10 and fastapi.

I did the same thing using python3.9 and it worked. It must be something with my local config, can you help, please?

The answer I get when calling the endpoint https://***********.execute-api.eu-west-3.amazonaws.com/development/api/health-check/ is {"message": "Internal server error"}.

The lambda's log says:

[ERROR] Runtime.ImportModuleError: Unable to import module 'main': No module named 'fastapi'
Traceback (most recent call last):

# main.py
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from mangum import Mangum

app = FastAPI()

app.add_middleware(CORSMiddleware, allow_origins="*", allow_credentials=True, allow_methods=["*"], allow_headers=["*"])

@app.get("/api/health-check/")
def health_check():
    return {"message": "OK"}

handle = Mangum(app)

Here is what I do:

$ poetry init
.....
Compatible Python versions [^3.9]:  3.10

Would you like to define your main dependencies interactively? (yes/no) [yes] no
Would you like to define your development dependencies interactively? (yes/no) [yes] no
Generated file
.....
[tool.poetry.dependencies]
python = "3.10"

$ poetry add --dev pytest pytest-cov black isort flake8 bandit
The currently activated Python version 3.9.9 is not supported by the project (3.10).
Trying to find and use a compatible version.

Poetry was unable to find a compatible version. If you have one, you can explicitly use it via the "env use" command.

On my machine I have 3 versions of python:

$ python3 --version
Python 3.9.9

$ python3.10 --version
Python 3.10.11

Editing the python version in pyproject.toml seams to unlock the situation:

[tool.poetry.dependencies]
python = "~3.10"

$ poetry add --dev pytest pytest-cov black isort flake8 bandit
The currently activated Python version 3.9.9 is not supported by the project (~3.10).
Trying to find and use a compatible version.
Using python3.10 (3.10.11)
Creating virtualenv test-sls-deploy-md9kc90P-py3.10 in /Users/costin/Library/Caches/pypoetry/virtualenvs
.....

$ poetry add fastapi uvicorn httpx
The currently activated Python version 3.9.9 is not supported by the project (~3.10).
Trying to find and use a compatible version.
Using python3.10 (3.10.11)
Using version ^0.95.1 for fastapi
.....

Then I deploy with

$ sls deploy --stage development --verbose

I get a warning that python3.10 is not in the list of expected runtime environments, but it deploys correctly.

Here is the serverless.yml file:

service: test-sls-deploy-api

frameworkVersion: '3'
useDotenv: true

provider:
  name: aws
  runtime: python3.10
  region: 'eu-west-3'
  stage: 'development'
  logRetentionInDays: 30

functions:
  TEST-DEPLOY:
    handler: main.handle
    memorySize: 512
    events:
      - http:
          path: /{proxy+}
          method: any
          cors:
            origin: ${env:ALLOWED_ORIGINS}
            maxAge: 60

custom:
  pythonRequirements:
    usePoetry: true
    noDeploy:
      - boto3  # already on Lambda
      - botocore  # already on Lambda

plugins:
  - serverless-python-requirements

I was expecting the plugin serverless-pytohn-requirements to handle the deployment of fastapi as it does with python3.9, but with python3.10 things do not happen as expected.

Do you see what could go wrong?

Answer 1

Not sure about the root cause of the problem, but doing that, solved the problem:

• Uninstall python3.9 and made python3.10 answer to python3.
• In pyproject.toml using python = "^3.10" instead of python = "~3.10"
• I do not know if that has an impact, but I've also created the .venv in project folder with

## poetry.toml
[virtualenvs]
in-project = true

In order to debug I used $ sls package and compare the .zip until the dependencies were there.

⚠️ There is still a warning from sls when deploying because it doesn't expect python3.10, but since AWS CloudFormation is ok with that everything is ok and the lambdas have the python3.10 runtime.