Search code examples
mongodbgorabbitmqcqrsevent-sourcing

How can I avoid repeating type definitions in an event sourced, event-driven architecture with CQRS?

The Situation

To better understand the concepts of CQRS, event sourcing, and asynchronous service communication, I've been building a small system using Go, MongoDB, and RabbitMQ. This includes the following:

  • Command API: exposes an API to accept and process commands then write events to a MongoDB collection (called "events")
  • Event Publisher: watches the "events" collection for changes and publishes those to RabbitMQ
  • Event Consumer: receives events from RabbitMQ and uses them to update a read-optimized MongoDB collection
  • Query API: exposes an API to return data from the materialized collection

(I envision a similar set of applications being repeated for each microservice in the system, all communicating asynchronously via RabbitMQ)

The Problem

I feel I have a decent grasp of these concepts at a high level, however in building this system I've run into some trouble with the details. I've found that I need to either:

  1. Use BSON as the RabbitMQ payload to make things "simple" for the Event Publisher - this feels like MongoDB is leaking into the rest of the design, since I wouldn't have chosen BSON otherwise

  2. Make the Event Publisher aware of all event types so it can properly translate a stored event from BSON to JSON (e.g. rewriting dates and UUIDs as strings) for publishing to the message bus - this seems to smell, as the various event types would need to be defined in no fewer than three separate places (command side, query side, and event publisher).

Questions

  1. Is this type of problem a normal occurrence for a design using CQRS + Event Sourcing + Message Bus, or is my approach fundamentally flawed?

  2. If this is par for the course, which of the above two options would be better to use in a production setting?

My searches have not turned up anything about this issue, but if you know of an article or blog post that addresses it that would probably be plenty.

Solution

  • There is the concept of event upgraders. The purpose of event upgraders is to transform events coming from the event store as a non-typed data structure into a typed data structure before publishing them (also before aggregates hydration). Those are often used in versioning. Remember more the system grows more are the chances your event types changes so you won't be able to just deserialize from json/bson.

    Once you're events are typed correctly serialize them in json to pass them into the bus and deserialize them back on the other side.

    event types would need to be defined in no fewer than three separate places (command side, query side, and event publisher)

    Just define your event types in a shared library.

    Also, it's not related but you really don't need a bus. They cause more problems when it comes to rebuilding projections since they don't keep track of the last read event. Greg Young talked about it somewhere on YouTube (should not be hard to find).