Search code examples
apache-kafkakafka-topic

How to update data in Kafka/Kafka stream?

Lets suppose there is Kafka topic orders. Data is stored in JSON format:

{
   "order_id": 1,
   "status": 1
}

Status defines status of order (pending - 1, completed - 2).

How to change it on completed when it is finished?

As I know Kafka topic immutable and I can not change message JSON, just create a new message with chnaged value, right?

Solution

  • If your order changes state, a process that is changing the state should generate a new message with the new state in the topic. The kafka streams application can react on new messages, do transformations aggregations or similar and output the modified/aggregated messages in new topics... So you need a kafka producer that when the order state changes, produces a message to the order topic.