I have an AWS SQS (Standard Queue) which listens to third party SNS. I have a lambda setup which has SQS trigger with Batch size 10000 and Batch window 300. My SQS receives approx. 150 messages at a time but lambda gets triggered in batches of 20-30 messages at a time even i configured Batch size 10000. I don't understand why this is happening... even the SQS have enough messages and enough time (300 seconds Batch window) to fill the batch, its not doing it.
I googled for the issue and found that maximum payload size for lambda can be 6MB. I checked my message and its approx. 2.5 KB per message. so 30*2.5 = 75 KB only and not touching the limit 6MB.
Additionally, I suspected lambda concurrency so i have set it up to value 1 only. so no parallel lambda instances.
Can somebody help me to understand where the problem is please?
Lambda uses five parallel long-polling connections to check your queue. So if you have 150 msgs, each connection gets about 30 msgs, exactly explaining what you see.
Sadly, you can't change the number of these connections. There are always five.