How to install external modules in a Python Lambda Function created by AWS CDK?

Question

I'm using the Python AWS CDK in Cloud9 and I'm deploying a simple Lambda function that is supposed to send an API request to Atlassian's API when an Object is uploaded to an S3 Bucket (also created by the CDK). Here is my code for CDK Stack:

from aws_cdk import core
from aws_cdk import aws_s3
from aws_cdk import aws_lambda
from aws_cdk.aws_lambda_event_sources import S3EventSource


class JiraPythonStack(core.Stack):
    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # The code that defines your stack goes here
        jira_bucket = aws_s3.Bucket(self,
                                    "JiraBucket",
                                    encryption=aws_s3.BucketEncryption.KMS)

        event_lambda = aws_lambda.Function(
            self,
            "JiraFileLambda",
            code=aws_lambda.Code.asset("lambda"),
            handler='JiraFileLambda.handler',
            runtime=aws_lambda.Runtime.PYTHON_3_6,
            function_name="JiraPythonFromCDK")

        event_lambda.add_event_source(
            S3EventSource(jira_bucket,
                          events=[aws_s3.EventType.OBJECT_CREATED]))

The lambda function code uses the requests module which I've imported. However, when I check the CloudWatch Logs, and test the lambda function - I get:

Unable to import module 'JiraFileLambda': No module named 'requests'

My Question is: How do I install the requests module via the Python CDK?

I've already looked around online and found this. But it seems to directly modify the lambda function, which would result in a Stack Drift (which I've been told is BAD for IaaS). I've also looked at the AWS CDK Docs too but didn't find any mention of external modules/libraries (I'm doing a thorough check for it now) Does anybody know how I can work around this?

Edit: It would appear I'm not the only one looking for this.

Here's another GitHub issue that's been raised.

Answer 1

It is not even necessary to use the experimental PythonLambda functionality in CDK - there is support built into CDK to build the dependencies into a simple Lambda package (not a docker image). It uses docker to do the build, but the final result is still a simple zip of files. The documentation shows it here: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#bundling-asset-code ; the gist is:

new Function(this, 'Function', {
  code: Code.fromAsset(path.join(__dirname, 'my-python-handler'), {
    bundling: {
      image: Runtime.PYTHON_3_9.bundlingImage,
      command: [
        'bash', '-c',
        'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output'
      ],
    },
  }),
  runtime: Runtime.PYTHON_3_9,
  handler: 'index.handler',
});

I have used this exact configuration in my CDK deployment and it works well.

And for Python, it is simply

aws_lambda.Function(
    self,
    "Function",
    runtime=aws_lambda.Runtime.PYTHON_3_9,
    handler="index.handler",
    code=aws_lambda.Code.from_asset(
        "function_source_dir",
        bundling=core.BundlingOptions(
            image=aws_lambda.Runtime.PYTHON_3_9.bundling_image,
            command=[
                "bash", "-c",
                "pip install --no-cache -r requirements.txt -t /asset-output && cp -au . /asset-output"
            ],
        ),
    ),
)