Does it make sense to use Apache Kafka for this Scenario?

Question

There are several applications which have to be integrated together and they have to exchange Issues. So one of them will get the issue and then do something and later on change the Status of this Issue. And the other applications which could be involved to this Issue should get the new Information. This continues until the Issue reaches the final Status Closed. The Problem is the Issue have to be mapped, because these applications do not all support the same Data Format.

1. I'm not sure whether to send the whole Issue always or just the new Status as an Event.
2. How does Kafka Support Data Transformation?
3. What if my Issue has an attachment?(>5MB)

Thanks for your advice

Answer 1

Yes it does make sense.

Kafka can do transformations through both the Kafka Streams API, and KSQL which is a streaming SQL engine built on top of Kafka Streams.

Typically Kafka is used for smaller messages; one pattern to consider for larger content is to store it in an object store (e.g. S3, or similar depending on your chosen architecture) and reference a pointer to it in your Kafka message.