I have a SNS topic t1 that is subscribed into a SQS queue q1. I have a NodeJS process p1 that publishes to t1, process p2 that subscribes from q1. I also have a process p3 that writes to q1 directly.
Suppose, data is populated when p2 reads from the queue. Then, while the following snippet works with p1 as the writer to the queue, I get a JSON parser error with p3.
for (var i = 0; i < data.Messages.length; i++) {
var message = data.Messages[i];
let messageBody = JSON.parse(message.Body)
let payload = JSON.parse(messageBody.Message)
The only way I could process data from p3 working is to not have JSON.parse(messageBody.Message) and process messageBody directly. Thus, it seems that, the message structure as received in a queue is different if the writer was a SNS topic subscription or a SQS writer.
Can you please advise if I can have a single NodeJS application that can process data from SQS regardless of what wrote into that queue?
Not 100% sure if this will solve all your issues, but as a first step at least you probably want to do enable 'Raw Message Delivery'
https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html
In this way, the messages that SNS puts in the queue won't have any additional properties added, and thus should match messages that are put in directly.