I have this Lambda (function URL) that works as expected.
def lambda_handler(event,context):
raw_query_string = event['rawQueryString']
dynamodb = boto3.resource('dynamodb')
dt = datetime.datetime.now()
seq = dt.strftime("%y%m%d%H%M%S")
table = dynamodb.Table('activitylg')
response = table.put_item(
Item={
"id": int(raw_query_string[:10]),
"dt": int(seq),
"code": int(raw_query_string[10:]),
}
)
But sometimes it adds a duplicate record. For e.g.
2023-08-26 08:39:44,2,0
2023-08-26 13:38:44,1,0
2023-08-26 14:44:55,2,0
2023-08-26 14:44:57,2,0
2023-08-27 08:01:40,2,1
2023-08-27 14:58:07,2,0
2023-08-27 14:58:08,2,0
2023-08-28 07:12:23,2,1
2023-09-20 11:39:57,2,1
2023-09-22 09:01:27,2,1
2023-10-03 08:30:33,2,1
2023-10-03 08:30:35,2,1
2023-10-03 08:31:52,1,1
2023-10-03 08:33:15,4,1
2023-10-03 08:33:36,3,1
There are 2 dupes in the above list.
2023-08-26 14:44:57,2,0
2023-10-03 08:30:35,2,1
I thought the application is (by mistake) sending the same record again after 2 seconds. But I guess it is something related to "cold start" problem.
How do I make sure that this is not something related to dynamoDB or lambda?
Update:
This is how the first request looks like in cloudwatch.
2023-08-26T14:44:54.160+05:30
INIT_START Runtime Version: python:3.11.v9 Runtime Version ARN: arn:aws:lambda:us-east-1::runtime:dabfb8de4b8bf97622f4c7eb46aed5a92c68ee3b72a98d1682a2dead82f52d15
2023-08-26T14:44:54.424+05:30
START RequestId: 4c792dc6-3cc9-4fd1-ac04-0341411fd5a1 Version: $LATEST
2023-08-26T14:44:56.226+05:30
END RequestId: 4c792dc6-3cc9-4fd1-ac04-0341411fd5a1
2023-08-26T14:44:56.226+05:30
REPORT RequestId: 4c792dc6-3cc9-4fd1-ac04-0341411fd5a1 Duration: 1802.76 ms Billed Duration: 1803 ms Memory Size: 128 MB Max Memory Used: 74 MB Init Duration: 263.11 ms
And this is duplicate request that was not initiated by me.
2023-08-26T14:44:57.412+05:30
START RequestId: e65b0ac1-bca3-486b-b767-8f96e6596794 Version: $LATEST
2023-08-26T14:44:57.649+05:30
END RequestId: e65b0ac1-bca3-486b-b767-8f96e6596794
2023-08-26T14:44:57.649+05:30
REPORT RequestId: e65b0ac1-bca3-486b-b767-8f96e6596794 Duration: 236.94 ms Billed Duration: 237 ms Memory Size: 128 MB Max Memory Used: 75 MB
I will check the code (kotlin) again. But I think there is no problem there because it works as expected most of the times. There are very rare cases of dupes.
I don't see any items the same, they all have millisecond difference. In DynamoDB if you can put a condition on the item not existing before you write it, it'll allow you to avoid duplicates.
As for why it's happening, it's not clear what's triggering your Lambda function, but you should check to make sure it only calls once. There are also SDK powertool kits for Lambda which can help you ensure idempotency.