Search code examples
rabbitmqamqpfifo

To be sure about concurrency, same group of works in multiple queues (FIFO)


I have a question about multi consumer concurrency. I want to send works to rabbitmq that comes from web request to distributed queues. I just want to be sure about order of works in multiple queues (FIFO). Because this request comes from different users eech user requests/works must be ordered.

I have found this feature with different names on Azure ServiceBus and ActiveMQ message grouping.

Is there any way to do this in pretty RabbitMQ ?

I want to quaranty that customer's requests must be ordered each other. Each customer may have multiple requests but those requests for that customer must be processed in order. I desire to process quickly incoming requests with using multiple consumer on different nodes. For example different customers 1 to 1000 send requests over 1 millions. If I put this huge request in only one queue it takes a lot of time to consume. So I want to share this process load between n (5) node. For customer X 's requests must be in same sequence for processing


Solution

  • RabbitMQ Consistent Hash Exchange Type

    We are using RabbitMQ and we have found a plugin. It use Consistent Hashing algorithm to distribute messages in order to consistent keys.

    For more information about Consistent Hashing ;

    https://en.wikipedia.org/wiki/Consistent_hashing

    https://www.youtube.com/watch?v=viaNG1zyx1g

    You can find this plugin from rabbitmq web page

    plugin : rabbitmq_consistent_hash_exchange

    https://www.rabbitmq.com/plugins.html