Event Sourcing and CQRS: Handle concurrency with multiple "read model consumers" instances

Question

I'm implementing a solution based on Event Sourcing and CQRS patterns.

In my use case I have:

• A WRITE microservice: Where events are appended to a particular Stream (let's call it X Stream) stored on an EventStoreDB instance.
• A READ microservice: Subscribed to the X Stream of the EventStoreDB, which consumes the appended events and store the projected model in a MongoDB instance.

In a simple scenario, where there is a single instance of the READ microservice, all works as expected:

1. An event is appended to the X Stream in the EventStoreDB
2. The single instance of the READ microservice consumes the event and stores the projected model on the MongoDB instance

Now suppose that you want to scale out the READ microservice (the event's consumer) to two or more instances. This is what will happen:

1. An event is appended to the X Stream in the EventStoreDB
2. Each replica of the READ microservice consumes the event and tries to store the projected model on the MongoDB instances, corrupting the READ model (because of the concurrent write).

Is there a way to handle this scenario?

Answer 1

Usually there is 1 active process with the catch-up subscription doing update to your read model.

And eventually a second one on stand-by if that first process should stop unexpectedly

When that becomes too slow , you can have multiple processes and partition them in such a way that they would handle a specific set of documents on the same target store.