Search code examples
dockeraws-lambdaaws-cloudformationamazon-sqs

How to access the SQS event that triggered a Lambda deployed as a Docker image?


I currently have a simple Lambda that's triggered by a SQS queue and prints out the event that triggered it. It's deployed with CFN and it's defined as:

PrintMessage:
    Type: AWS::Lambda::Function
    Properties:
      Handler: index.handler
      Runtime: python3.8
      FunctionName: !Sub printMessage-${Environment}
      MemorySize: 1024
      Role: !Sub arn:aws:iam::${AWS::AccountId}:role/service-role/printMessageRole-${Environment}
      Timeout: 900
      Code:
        ZipFile: |
          import os
          import json

          def handler(event, context):
            print("Event: {}".format(event))

PrintMessageEventSource:
    Type: AWS::Lambda::EventSourceMapping
    Properties:
      BatchSize: 1
      Enabled: true
      EventSourceArn: !Sub arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:printMessageQueue-${Environment}
      FunctionName: !Sub printMessage-${Environment}

This flow works fine - anything that is written to printMessageQueue triggers the Lambda and its content is printed.

My problem is that I need to use another component's logic which is more complex and contains several dependencies and is already deployed as a Docker image to ECR. What I have so far is:

PrintMessage:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ImageUri: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/component:${ImageTag}
      FunctionName: !Sub printMessage-${Environment}
      ImageConfig:
        EntryPoint:
          - python3
          - print_message.py
      MemorySize: 1024
      PackageType: Image
      Role: !Sub arn:aws:iam::${AWS::AccountId}:role/service-role/printMessageRole-${Environment}
      Timeout: 900

It works as expected, running python3 print_message.py inside a Docker container.

Is there any way to access the SQS event that triggered the Lambda, similarly to def handler(event, context) line, but inside the print_message.py script:

import os
import json

event = ???
print("Event: {}".format(event))

Solution

  • UPDATED: To include the "how to know the source"

    My understanding is that you would like to migrate docker code that you have created to Lambda function. You would like to reuse the container code, and invoke it as a Lambda.

    If my assumption is correct then. Your container needs to run the Lambda Runtime API. AWS provides base docker images with the runtime pre-installed.

    Please note the requirements when deploying a container image to Lambda, the documentation states:

    • The container image must implement the Lambda Runtime API.
    • The container image must be able to run on a read-only file system. Your function code can access a writable /tmp directory with 512 MB of storage.
    • The default Lambda user must be able to read all the files required to run your function code. Lambda follows security best practices by defining a default Linux user with least-privileged permissions. Verify that your application code does not rely on files that other Linux users are restricted from running.
    • Lambda supports only Linux-based container images.

    The handler that you used in the zip file is still required, that is set using the container image settings :

    Lambda supports the following container image settings in the Dockerfile, the AWS documentation outlines the following container settings:

    • ENTRYPOINT – Specifies the absolute path to the entry point of the application.

    • CMD – Specifies parameters that you want to pass in with ENTRYPOINT.

    • WORKDIR – Specifies the absolute path to the working directory.

    • ENV – Specifies an environment variable for the Lambda function.

    Here is a great blog post that describes the difference between CMD and ENTRYPOINT.

    The following blog post explains how to create a docker container that you can deploy as a Lambda function.

    Once your container is deployed to ECR, you can then use CloudFormation like you have to deploy the container in ECR as a Lambda Function using: AWS::Lambda::Function.

    You set the package type to Image if the deployment package is a container image. For a container image, the code property must include the URI of a container image in the Amazon ECR registry. You do not need to specify the handler and runtime properties, as this is set with the Container Settings mentioned above.

    You can then use CloudFormation to also create a SQS Queue that invokes the Lambda Function AWS::Lambda::EventSourceMapping.

    More information is available here.

    The event handler for a standard node.js Lambda function deployed as a zip or via the console will have three parameters passed:

    • event
    • context
    • callback

    This structure is the same for other languages.

    • Python: def lambda_handler(event, context):
    • Java: public String handleRequest(Map<String,String> event, Context context)
    • .Net: public async Task Handler(ILambdaContext context)
    • GoLang: func HandleRequest(ctx context.Context, name MyEvent) (string, error) {
    • Ruby: def handler(event:, context:)

    The event function returns information about the calling event, in this case SQS. The structure is dependent on the calling service. For SQS it is:

    "Records": [
            {
                "messageId": "5f3feef3-xxxx-xxxx-xxxx-xxxxxxx",
                "receiptHandle": "some handle",
                "body": "body from SQS message",
                "attributes": {
                    "ApproximateReceiveCount": "1",
                    "SentTimestamp": "1628768326353",
                    "SenderId": "senderid",
                    "ApproximateFirstReceiveTimestamp": "1628768326358"
                },
                "messageAttributes": {},
                "md5OfBody": "checksum",
                "eventSource": "aws:sqs",
                "eventSourceARN": "arn:aws:sqs:[REGION]:[AccountID]:[FunctionName]",
                "awsRegion": "REGION"
            }
        ]
    

    From the event you can check the event source field within Records to determine what the event source is.

    The process does not change with a Docker deployment, here you will still have a handler that receives the event and the context. The CMD of the Docker image points to the handler.

    This is shown in the following blog posts:

    The runtime interface client in your container image manages the interaction between Lambda and your function code. The Runtime API, along with the Extensions API, defines a simple HTTP interface for runtimes to receive invocation events from Lambda and respond with success or failure indications.