Chat bots: ensuring serial processing of messages on a per-conversation basis in clustered environment

Question

In the context of writing a Messenger chat bot in a cloud environment, I'm facing some concurrency issues.

Specifically, I would like to ensure that incoming messages from the same conversation are processed one after the other.

As a constraint, I'm processing the messages with workers in a Cloud environment (i.e the worker pool is of variable size and worker instances are potentially short-lived and may crash). Also, low latency is important.

So abstracting a little, my requirements are:

• I have a stream of incoming messages
• each of these messages has a 'topic key' (the conversation id)
• the set of topics is not known ahead-of-time and is virtually infinite
• I want to ensure that messages of the same topic are processed serially
• on a cluster of potentially ephemeral workers
• if possible, I would like reliability guarantees e.g making sure that each message is processed exactly once.

My questions are:

1. Is there a name for this concurrency scenario?.
2. Are there technologies (message brokers, coordination services, etc.) which implement this out of the box?
3. If not, what algorithms can I use to implement this on top of lower-level concurrency tools? (distributed locks, actors, queues, etc.)

Answer 1

I don't know of a widely-accepted name for the scenario, but a common strategy to solve that type of problem is to route your messages so that all messages with the same topic key end up at the same destination. A couple of technologies that will do this for you:

• With Apache ActiveMQ, HornetQ, or Apache ActiveMQ Artemis, you could use your topic key as the JMSXGroupId to ensure all messages with the same topic key are processed in-order by the same consumer, with failover
• With Apache Kafka, you could use your topic key as the partition key, which will also ensure all messages with the same topic key are processed in-order by the same consumer

Some message broker vendors refer to this requirement as Message Grouping, Sticky Sessions, or Sticky Message Load Balancing.

Another common strategy on messaging systems with weaker delivery/ordering guarantees (like Amazon SQS) is to simply include a sequence number in the message and leave it up to the destination to resequence and request redelivery of missing messages as needed.