Search code examples
pythonamazon-web-servicesaws-lambdaterraformamazon-sqs

Number of deleted messages is twice as many as received messages


I created SQS service with terraform

resource "aws_sqs_queue" "ses_queue" {
  name                       = "ses_queue"
  message_retention_seconds  = 86400
  receive_wait_time_seconds  = 1
  visibility_timeout_seconds = 15
}

resource "aws_lambda_event_source_mapping" "send_email_message" {
  event_source_arn = aws_sqs_queue.ses_queue.arn
  function_name    = aws_lambda_function.send_email_message.function_name
  batch_size       = 5
}

I am sending emails using lambda function

for record in event.get("Records"):
    receipt_handle = record.get("receiptHandle", "")
    request_body = record.get("body")
    response = send_email(request_body)
    if response:
        sqs_client.delete_message(QueueUrl=constants.SES_QUEUE_URL, ReceiptHandle=receipt_handle)

I am wondering why number of deleted messages is twice as many as received messages enter image description here

enter image description here


Solution

  • Fixed it with reporting batch item failures

    terraform

    resource "aws_lambda_event_source_mapping" "send_email_message" {
      event_source_arn         = aws_sqs_queue.ses_queue.arn
      function_name            = aws_lambda_function.send_email_message.function_name
      batch_size               = 5
      function_response_types  = ["ReportBatchItemFailures"]
    }
    

    lambda

    reprocess_messages = []
    for record in event.get("Records"):
        receipt_handle = record.get("receiptHandle", "")
        message_id = record.get("messageId", "")
        request_body = record.get("body")
        response = send_email(request_body)
        if not response:
            reprocess_messages.append({"itemIdentifier": message_id})
    
    return {"batchItemFailures": reprocess_messages}