I'm architecting RabbitMQ into our solution, and I'm curious on how to handle processing and acknowledging messages efficiently, while still performing "real" work in the consumer code that can range from 5-10 seconds. (More work than the samples ever delve into).
Above is an example of what I'm aiming to process. A message in my twitter.tweet_cmd_q
queue that has all the parameters needed in the message body, for the consumer to make the actual Twitter API request, and save those results to the DB.
However, I'm running into two problems here:
Would RPC calls benefit me at all here, in this situation that involves querying the data and saving it to the DB?
Is the best way to deal with this scalability, just to create more worker instances for round robin handling?
Yes, it is normal for the consumer code to do everything it needs with the message before ack'ing it. That's how you manage the message visibility to other workers.
No, keep it simple, just do what you need to do in the worker.
Sort of, it's not really round robin, just create more workers, and have them subscribe to the queue. Each worker will poll the queue, find messages, and execute on them.