Search code examples
pythonamazon-web-servicesaws-lambdafastapi

Unable to import module 'src.main': No module named 'dependencies' when uploading FastAPI zipped project to AWS Lambda

I am developing a simple FastAPI app and I want to deploy it on AWS Lambda.

This is the structure of the project:

structure

And I ran this command for creating the dependencies directory:

pip3 install -r requirements.txt --platform manylinux2014_x86_64 --target=dependencies --implementation cp --python-version 3.12 --only-binary=:all: --upgrade openai

Then adding the content of dependencies to aws_lambda_artifact.zip:

(cd dependencies; zip ../aws_lambda_artifact.zip -r .)

And finally adding to the .zip archive the content of src directory:

zip aws_lambda_artifact.zip -u -r src

You can see here that the archive contains the src directory:

src

And this is its content:

enter image description here

Unfortunately when I upload the .zip archive into AWS Lambda, whose configuration is:

lambda

I get:

{
  "errorMessage": "Unable to import module 'src.main': No module named 'dependencies'",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "",
  "stackTrace": []
}

In the "test" panel of the console. What's wrong?

In case you need it, this is the main.py file:

from fastapi import FastAPI
from mangum import Mangum

from src import models
from src.database import engine
from src.routers import blog, user, authentication

app = FastAPI()
handler = Mangum(app)

models.Base.metadata.create_all(bind=engine)

app.include_router(blog.router)
app.include_router(user.router)
app.include_router(authentication.router)

And this is the requirements.txt file:

fastapi==0.114.2
mangum==0.17.0
SQLAlchemy==2.0.34
passlib==1.7.4
bcrypt==4.2.0
python-jose==3.3.0
python-multipart==0.0.9

This is the full log output:

/var/task/pydantic/_internal/_config.py:341: UserWarning: Valid config keys have changed in V2:
* 'orm_mode' has been renamed to 'from_attributes'
warnings.warn(message, UserWarning)
[ERROR] Runtime.ImportModuleError: Unable to import module 'src.main': No module named 'dependencies'
Traceback (most recent call last):INIT_REPORT Init Duration: 1323.45 ms Phase: init Status: error   Error Type: Runtime.Unknown
/var/task/pydantic/_internal/_config.py:341: UserWarning: Valid config keys have changed in V2:
* 'orm_mode' has been renamed to 'from_attributes'
warnings.warn(message, UserWarning)
[ERROR] Runtime.ImportModuleError: Unable to import module 'src.main': No module named 'dependencies'
Traceback (most recent call last):INIT_REPORT Init Duration: 19347.21 ms    Phase: invoke   Status: error   Error Type: Runtime.Unknown
START RequestId: 6ae13d71-0d1f-428a-8793-e79ee0590f81 Version: $LATEST
END RequestId: 6ae13d71-0d1f-428a-8793-e79ee0590f81
REPORT RequestId: 6ae13d71-0d1f-428a-8793-e79ee0590f81  Duration: 19376.13 ms   Billed Duration: 19377 ms   Memory Size: 128 MB Max Memory Used: 104 MB Status: error   Error Type: Runtime.Unknown

This is the template API Gateway AWS Proxy I am using for testing the API:

{
  "body": "eyJ0ZXN0IjoiYm9keSJ9",
  "resource": "/{proxy+}",
  "path": "/blogs",
  "httpMethod": "GET",
  "isBase64Encoded": true,
  "queryStringParameters": {
    "foo": "bar"
  },
  "multiValueQueryStringParameters": {
    "foo": [
      "bar"
    ]
  },
  "pathParameters": {
    "proxy": "/blogs"
  },
  "stageVariables": {
    "baz": "qux"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "en-US,en;q=0.8",
    "Cache-Control": "max-age=0",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "US",
    "Host": "1234567890.execute-api.us-east-1.amazonaws.com",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Custom User Agent String",
    "Via": "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA==",
    "X-Forwarded-For": "127.0.0.1, 127.0.0.2",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "multiValueHeaders": {
    "Accept": [
      "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"
    ],
    "Accept-Encoding": [
      "gzip, deflate, sdch"
    ],
    "Accept-Language": [
      "en-US,en;q=0.8"
    ],
    "Cache-Control": [
      "max-age=0"
    ],
    "CloudFront-Forwarded-Proto": [
      "https"
    ],
    "CloudFront-Is-Desktop-Viewer": [
      "true"
    ],
    "CloudFront-Is-Mobile-Viewer": [
      "false"
    ],
    "CloudFront-Is-SmartTV-Viewer": [
      "false"
    ],
    "CloudFront-Is-Tablet-Viewer": [
      "false"
    ],
    "CloudFront-Viewer-Country": [
      "US"
    ],
    "Host": [
      "0123456789.execute-api.us-east-1.amazonaws.com"
    ],
    "Upgrade-Insecure-Requests": [
      "1"
    ],
    "User-Agent": [
      "Custom User Agent String"
    ],
    "Via": [
      "1.1 08f323deadbeefa7af34d5feb414ce27.cloudfront.net (CloudFront)"
    ],
    "X-Amz-Cf-Id": [
      "cDehVQoZnx43VYQb9j2-nvCh-9z396Uhbp027Y2JvkCPNLmGJHqlaA=="
    ],
    "X-Forwarded-For": [
      "127.0.0.1, 127.0.0.2"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "resourceId": "123456",
    "stage": "prod",
    "requestId": "c6af9ac6-7b61-11e6-9a41-93e8deadbeef",
    "requestTime": "09/Apr/2015:12:34:56 +0000",
    "requestTimeEpoch": 1428582896000,
    "identity": {
      "cognitoIdentityPoolId": null,
      "accountId": null,
      "cognitoIdentityId": null,
      "caller": null,
      "accessKey": null,
      "sourceIp": "127.0.0.1",
      "cognitoAuthenticationType": null,
      "cognitoAuthenticationProvider": null,
      "userArn": null,
      "userAgent": "Custom User Agent String",
      "user": null
    },
    "path": "/blogs",
    "resourcePath": "/{proxy+}",
    "httpMethod": "GET",
    "apiId": "1234567890",
    "protocol": "HTTP/1.1"
  }
}

Amazon Q says:

The error indicates that the Lambda function failed to import the module 'src.main' because it could not find the dependencies module that 'src.main' relies on. This often occurs when a module import is missing or incorrectly specified in the function code.

Also:

1. Review your Lambda function code and ensure that the 'dependencies' module is correctly installed and available in the deployment package.

2. If the 'dependencies' module is not included in the deployment package, update your function code to include the missing module:
   - If using a virtual environment, install the 'dependencies' module and create a new deployment package with the updated virtual environment.
   - If not using a virtual environment, install the 'dependencies' module and include it in your deployment package.

3. After updating the deployment package with the missing 'dependencies' module, update the Lambda function code by uploading the new deployment package.

4. If the issue persists after updating the deployment package, review the Lambda function configuration and ensure that the correct handler is specified for the 'src.main' module.

5. If the issue still persists, review the Lambda function logs for any additional errors or information that could help identify the root cause.
Solution

  • I slipped and wrote

    from dependencies.fastapi import FastAPI

    instead of

    from fastapi import FastAPI

    And for this reason the engine was expecting to be provided a dependencies directory containing the specified module.