Search code examples
spring-bootapache-kafkarabbitmqmessagebrokerapache-pulsar

Message Brokers - Multiple consumers with this same client ID


I've been considering some multiple consumer problem with my system and unfortunately I'm really stuck. I mean - I see some solutions, which are below, but they are propably not enought efficient. Let me introduce system:

User has their specific Id. User can be logged on diffrent devices - many mobiles, many browsers in this same time. When user is offline every message which user has got should be provide when user is online. When user is online, should be kept informed on a regular basis about messages. Every user, when is online is connected by WebSocket.

So I have been thinking about message brokers from this pool - rabbitmq, kafka, apache pulsar ( All system above will be in Java ). And this is my thoughs about this:

  1. Rabbitmq - Every device gets their own queue associated with client Id. But here i see some troubles. For example - user gonna log in 4 browsers and each will get new queue ( of course I assume that some not used queue will be deleted after time but this solution can be overloaded if somebody just wanna do this ).

  2. One queue with marker for every user which messages were consumed - I tried implement this with apache Pulsar but my every attempt was running out with created new consumer (not continue as the same consumer) - Maybe I can't use this API?

  3. Apache kafka - groups and partitions? Its similiar to point 1.

I will be really gratefull for every single hint - If You see some better solutions with other technology just let me know - I will adjust to this.

( If it matters - Java and SpringBoot are core of this )


Solution

  • I can respond for the apache pulsar part - you need to set SubscriptionName in consumer to be equal to your UserId, this will ensure messages to be consumed starting from the last acknowledged one for that user.