Search code examples
apache-pulsar

Can Pulsar support multiple schemas on a single topic


I'm interested in the Schema functionality provided by Pulsar, but I want to validate an assumption before moving forward.

Let's say I have an inventory management system with a single topic called inventorymgmt. I have a producer than produces several events such as InventoryItemAdded and InventoryItemRenamed.

All of the inventory-related events are posted to the same inventorymgmt topic.

It looks to me that if I want to use the schema functionality, it's limited to a single message type per topic. The implication is that I would need separate topics for each type of event if I want to leverage the schema support.

I've tried and failed to do something like:

var producer1 = await client.NewProducer(Schema.JSON<InventoryItemAdded>()).Topic("inventorymgmt");

var producer2 = await client.NewProducer(Schema.JSON<InventoryItemRenamed>()).Topic("inventorymgmt");

The above throws an IncompatibleSchemaException.

It makes sense that it would work that way, but I just want to confirm that my assumption is correct and that it's not just a matter of missing some type of configuration setting.


Solution

  • Yes, there is a single schema per topic in Pulsar. All events sent to the topic must conform to the schema. To have different event types, you should use multiple topics each with their own schema.

    Alternatively, you could define a generic schema that works for multiple event types. In your example, you could have a generic Item schema with an Action field that could be set to Added, Renamed, etc.