Search code examples
rabbitmqmicroservices

Synchronize microservices with events (RabbitMQ)

We are using a microservice architecture to implement Web-APIs in nodejs. Every service exposes HTTP endpoints so the app / website can interact with it. To synchronize the different databases, we are currently using RabbitMQ. A microservice can publish a message on a fanout exchange and every subscribed microservice receives the message.

There are two problems with this architecture.

  1. What if we want to add a second instance of a microservice (for loadbalancing purposes etc.). If the second service would subscribe to the same fanout exchange, the messages would be consumed two times.

  2. Either acknowledgments do not work with fanout exchanges, or I'm doing something wrong. When I publish a message on a fanout exchange without subscribers, the messages disappears immediately without being acked.

This leads me to the my question. Is RabbitMQ a good choice for microservice synchronization or should we change our architecture. Here is a short example of how I would like it to work:

  1. The user creates a new account
  2. The auth-mc inserts the user in its database and publishes a 'user.created' event
  3. a1-mc, a2-mc (same mc, just loadbalanced) and b1-mc are subscribers of the exchange. Either a1 or a2, as well as b1 receive the event and insert the user in their respective database
  4. The event is only removed from each microservices queue, after its acknowledged

This way I can be sure, that every microservice (loadbalanced or not) receives the message one time. Can such a pattern even be implemented using RabbitMQ?

EDIT: Also looking for good literature about microservices if there are any suggestions.

Solution

  • Let's use topic exchange instead of fanout for your purpose. Only one consumer will receive the message instead of all of them. You could route your messages based on routing_key params for different consumers. For instance, you have an exchange. You have three different queues bound to this exchange with the same routing key. Your message will be duplicated for each queue! Your consumers from different microservices could read the message separately and do what they need. The message will be not dropped until you acknowledge it, but it's a good practice to push message with TTL.