Search code examples
asynchronousarchitecturemessage-queueservicebus

How can I report the status of an item in a message queue

I am planning to implement an asynchronous file generation module in my application using a message queue (RabbitMQ). When users input data through a web application UI, the data is put into the MQ. The data contains the ID of the requesting user as well. There is a worker application that polls the queue. The worker removes an item from the queue, processes the record, creates a file as a result, and stores the file into database. My requirement is that I want to show a grid to the user showing the status of all requests they have made, as shown in the following table

----------------------------
Report Name  |  Status      
----------------------------
Report 1     |  Not Started 
Report 2     |  Completed   
----------------------------

All the items still in the MQ should show the status as 'Not Started'. Theoretically, this will make me query all the messages in the MQ where User ID = current user ID. I think it may not be very efficient to peek all messages from MQ unlike we do in a SQL table. Basically, I am looking for various/better options to handle this the best way possible from a design/architecture perspective.

Solution

  • You need to create a shadow table somewhere in a database (SQL) then when you push an item in the queue add an entry in the database, when you pull the item from the queue update the status, and when you complete do it again. Yes, it is a bit of a headache to have a second database, but queues are designed to help you process messages sequentially, not update the status on all of them.