Search code examples
amazon-web-servicesamazon-sqsfifo

AWS SQS FIFO QUEUE: Unable to send message using loop in single run


I am using AWS SQS service and currently trying to insert the message in the queue and I am successfully able to insert the first message but not able to insert second one using loop... it returns the same message-id and sequence number

MessageId: '****-a938-4fdc-abed-2efb1e302a2a', SequenceNumber: '18848*******2174575616'

can anyone is able to help me below is code I am using:-

function insertMessage(data){
    var sqs = new AWS.SQS({apiVersion: '2012-11-05'});

        var params = {

           MessageGroupId:data.messageGroupId,
           MessageAttributes: {
                "operation": {
                    DataType: "String",
                    StringValue: "insert_comment"
                },
                "comment_id": {
                    DataType: "Number",
                    StringValue: ""+data.comment_id+""
                },
                "timestamp": {
                    DataType: "String",
                    StringValue: ""+data.created_on+""
                }
            },
            MessageBody: "New comment send ",

            QueueUrl: "******"
        };
        //console.log(params);
        sqs.sendMessage(params, function (err, data) {
            if (err) {
                console.log("Error", err);
                callback(err);
            } else {
                console.log("Success", data.MessageId);
                callback(data);
            }
        });
}

I am calling above function in loop.

And every time I am getting the same MessageId and SequenceNumber in response in the callback.

{ ResponseMetadata: { RequestId: '762a88d4-****-5298-***-dd3cb27f7dc1' },
  MD5OfMessageBody: '33d9f3a9*****bad5f72014ea836062',
  MD5OfMessageAttributes: '*****af0c13771494a53fdb2',
  MessageId: '*****-****-4fdc-abed-2efb1e302a2a',
  SequenceNumber: '188******174575616'

}


Solution

  • Apparently you don't add MessageDeduplicationId attribute which indicates that content based duplication is enabled. In this case, MessageDeduplicationId is automatically generated for you based on the message body content hash. Here, 5 minutes deduplication interval applies. If you want to add message to the queue with same message body and within the deduplication interval then you will need to generate a unique MessageDeduplicationId for each message.

    References

    If multiple messages are sent in succession to a FIFO queue, each with a distinct message deduplication ID, Amazon SQS stores the messages and acknowledges the transmission. Then, each message can be received and processed in the exact order in which the messages were transmitted.

    MessageDeduplicationId

    The token used for deduplication of sent messages. If a message with a particular message deduplication ID is sent successfully, any messages sent with the same message deduplication ID are accepted successfully but aren't delivered during the 5-minute deduplication interval.

    Deduplication configuration

    Enable content-based deduplication. This instructs Amazon SQS to use a SHA-256 hash to generate the message deduplication ID using the body of the message—but not the attributes of the message