Laravel and SQS - Several connections in one request

Question

I have a Laravel app (iOS API) that pushes data to SQS to be processed in the background. Depending on the request, we need to dispatch anywhere from 1 to 4 jobs to SQS. For example:

Dispatch a job to SQS, to be processed by a Worker:

• for connecting to a socket service (Pusher)
• for connecting to Apple's APNS service (Push Notifications)
• for sending data to Loggly for basic centralized request logging
• for storing analytics data in a SQL database (for the time being)

The problem is, we might have a feature like "chat", which is a pretty light request as far as server processing is concerned, however it needs to connect to SQS three times over to send:

• 1) Socket push to all the devices
• 2) Analytics processing
• 3) Centralized Request / Error Logging

In total, these connections end up doubling or tripling the time that the rest of request takes. ie. POSTing to /chat might otherwise take about 40-50ms, but with SQS, it takes more like 100 - 120ms

Should i be approaching this differently? Is there a way batch these to SQS so we only need to connect once rather than 3 times?

Answer 1

As Michael suggests in the comments, one way to do it is to use a sendMessageBatch request to send up to 10 messages to one single queue. When you're using multiple queues, and maybe even if you're not, there's also another approach.

If you can approach all the different messages as the same element, namely a notification of an action to an arbitrary receiver, you will find yourself in the fanout pattern. This is a commonly used pattern in which multiple receivers need to act on a single message, a single action.

Although AWS SQS doesn't natively support it, in combination with AWS Simple Notification Service you can actually achieve the same thing. Jeff Barr wrote a short blog post on the fanout setup using SNS and SQS a while ago (2012). It boils down to sending a notification to SNS that'll trigger messages to be posted on multiple SQS queues.