Search code examples
amazon-web-servicesaws-lambdaamazon-kinesisamazon-kinesis-kpl

Failed to put record in kinesis by lambda


Since I am new to lambda and kinesis so apologies in advance.

I have an SQS subscribed to SNS. A lambda polls the message from SQS and publish it to kinesis after XYZ processing.

I want to know if putting record to kinesis by lambda fails what is retry mechanism for it ?

Suppose it retries multiply by code retry strategy and again fails. Since lambda has not been able to put it to kinesis after multiple retries do we have any mechanism to get notified about failure of putting records in kinesis? Will it be considered as lambda not able to process the record, put it to SQS DLQ?

How often do it fails (e.g pushing a record to an SQS subscribed to an SNS have almost negligible failures)?


Solution

  • Assumptions: An aws lambda function is trying to add a record in aws kinesis stream.

    import boto3
    
    
    kinesis_client = boto3.resource
    attempt = 0
    
    while attempt < 5:
      attempt += 1
      try:
        # trying to add a record in a kniesis stream
        response = kinesis_client.put_record(
            StreamName='some_stream_1',
            Data=data,
            PartitionKey='sone_key_123',
        )
      except Exception as err:
        if attempt < 5:
          pass
        else:
          # TODO send an email notification using aws SES
          ses_client.send_email()