Seeing that some event sources send multiple event records in a batch1, and that a single Lambda may be made available to both to a multiple-record event sources (e.g. SQS, SES) and a single request event source (e.g. API Gateway), does anyone have any examples of handling both a single API request and multiple record SQS or other source events with the same Lambda?
This is a little more involved than limiting the batch size, since the event objects from different services don't always have a list of records, e.g. API Gateway sends a single request object event payload, rather than a list of records.
I have been able to figure out how to parse the events based on the eventSource key; however, I don't understand how Lambda responds to multiple-record events.
Is there one invocation record per event record, or is there a single invocation record, i.e. does the Lambda code run once per batch or multiple times?
Is my Lambda code responsible for forming the invocation record(s), or just the response payload?
How do I form my response payload(s) from a multiple-record event batch?
Do I respond to each record individually and build up a single payload with the responses for each individual record, then return the aggregate payload?
On one hand, I may just want the Lambda to do something and not return anything, such as an asynchronous invocation that isn't expecting a response. On the other hand, a synchronous API gateway request would expect a response, or another asynchronous invocation may expect responses to be published to another queue.
I am making the assumption that each record in a multiple-record SQS Queue event batch is equivalent to a single request event from API Gateway. Is this incorrect?
A Lambda is invoked once per event whether the event contains a single request, or a batch of multiple records as with SQS Queue invocations. Using AWS Lambda with Amazon SQS or as Alex DeBrie of Serverless Blog writes: "With the SQS / Lambda integration, a batch of messages succeeds or fails together."Using SQS with AWS Lambda and Serverless
You can code your Lambda function to process each record of a batch in a loop and perform some action for each. After all the records are processed, the function may return a response payload. This could be the string "Success" or another message or data or None. How you format the payload, and what data you return depends on what needs to be done with it. Alternatively, the Lambda function can raise an error, or exit without returning anything; however, in these instances, the messages in the batch will remain visible in the queue.
As far as I can tell, what you return in your response payload doesn't mean anything to SQS. Success or failure determines whether a retry is necessary or whether the batch of messages can be deleted or moved to a dead letter queue. You may find it useful to return a meaningful message or other data since this will be logged; however, SQS does not do anything with it.
If individual messages in a batch inform downstream processes, your Lambda code can transmit individual responses to another service, such as another SQS queue or an API, one at a time as the records are processed.