AWS lambda send batch messages to SQSClient timeout

Question

In my nodejs lambda function aws SQSClient send a batch request to a dlq queue

Why does my send request always fails with a timeout ?

const sqs = new SQSClient({ region: process.env.AWS_REGION ?? 'eu-west-3' });

export async function sendFailedEventsToDLQ<T>(messages: T[], context: Context, dlqName: string | undefined): Promise<void> {
  if (!messages.length) return;
  const entries: SendMessageBatchRequestEntry[] = messages.map((message) => {
    return { Id: dlqName, MessageBody: formatSQSMessage<T>(message) };
  });
  const params = new SendMessageBatchCommand({
    QueueUrl: getDLQUrl(context, dlqName),
    Entries: entries,
  });
  logger.info({ params }, 'Sending messages to dlq...');
  const response = await sqs.send(params).catch((error) => logger.error(error)); // <-- the line that is failing in timeout
  logger.info({ response }, 'Messages has been sent to dlq !');
}

function formatSQSMessage<T>(message: T): string {
  const foramtedMessage = {
    Message: JSON.stringify(message),
  };
  return JSON.stringify(foramtedMessage);
}

function getDLQUrl(context: Context, dlqName: string | undefined): string {
  if (!dlqName) throw new Error(Errors.ENV_VARIABLE_DLQ_NAME_UNDEFINED);
  const region = context.invokedFunctionArn.split(':')[3];
  const accountId = context.invokedFunctionArn.split(':')[4];
  return `https://sqs.${region}.amazonaws.com/${accountId}/${dlqName}`;
}

I've allowed sendMessageBatch in my serverless.ts file

{
        Effect: 'Allow',
        Action: ['SQS:SendMessage', 'SQS:SendMessageBatch', 'SQS:ReceiveMessage', 'SQS:DeleteMessage', 'SQS:GetQueueAttributes'],
        Resource: '*',
},

The exact error I have in my log is right after the params log

    2023-04-16T01:22:31.893+02:00   {"level":30,"time":1681600951855,"pid":8,"hostname":"169.254.55.108","params":{"middlewareStack":{},"input":{"QueueUrl":"https://sqs.eu-west-3.amazonaws.com/907108099392/mailing-webhook-service-update-pros-queue-dlq-prodv1","Entries":[{"Id":"mailing-webhook-service-update-pros-queue-dlq-prodv1","MessageBody":"{\"Message\":\"{\\\"event-data\\\":{\\\"recipient\\\":\\\"test@test.com\\\",\\\"event\\\":\\\"opened\\\"},\\\"dlqError\\\":{\\\"dlqName\\\":\\\"mailing-webhook-service-update-pros-queue-dlq-prodv1\\\",\\\"errorCode\\\":\\\"NotProcessedMessagesError\\\"}}\"}"}]}},"msg":"Sending messages to dlq..."}
    2023-04-16T01:22:41.834+02:00   2023-04-15T23:22:41.834Z df91f075-eba4-5606-ad66-acc0d80a7d39 Task timed out after 10.01 seconds 

my lambda config

export default {
  handler: `${getCurrentFolderPath(__dirname)}/handler.main`,
  timeout: 10,
  memorySize: 512,
  vpc: {
    securityGroupIds: VPC_SECURITY_GROUP_IDS,
    subnetIds: VPC_SUBNET_IDS,
  },
  events: [
    {
      sqs: {
        arn: {
          'Fn::GetAtt': ['${self:custom.updateProsQueue}', 'Arn'],
        },
        batchSize: 10,
        maximumBatchingWindow: 60,
      },
    },
  ],
};

Answer 1

When an AWS Lambda function is NOT connected to a VPC, it has direct access to the Internet. The Amazon SQS API endpoint is on the Internet, so the Lambda function is able to send API requests to the SQS service.

When the function IS connected to a VPC, it does not have a public IP address and is unable to communicate with the Internet. Therefore, it is not able to send a request to the SQS endpoint.

There are two options:

Option 1: Use a NAT Gateway

• Put the Lambda function in a private subnet
• Launch a NAT Gateway in the public subnet (charges apply)
• Route Internet-bound requests from the private subnet to the NAT Gateway

Option 2: Use an SQS VPC Endpoint

From Amazon SQS now Supports Amazon VPC Endpoints using AWS PrivateLink - Amazon Web Services:

AWS customers can now access Amazon Simple Queue Service (Amazon SQS) from their Amazon Virtual Private Cloud (Amazon VPC) using VPC endpoints, without using public IPs, and without needing to traverse the public internet.

This allows the Lambda function to send the request to the Amazon SQS endpoint without requiring Internet connectivity. I would recommend this option.