Search code examples
apache-kafkaksqldbktable

Implement SQL update with Kafka

How can I implement an update of the object that I store in Kafka topic / Ktable?

I mean, if I need not the replacement of the whole value (which compacted Ktable would do), but a single field update. Should I read from the topic/Ktable, deserialize, update the object and then store the new value at the same topic/KTable?

Or should I join/merge 2 topics: one with the original value and the second with the update of the field?

What would you do?

Solution

  • Kafka (and RocksDB) stores bytes; it cannot compare nested fields as through they are database columns. In order to do so would require deserialization anyway

    To update a field, you need to construct and post that whole value; a JOIN will effectively do the same thing

    Related - Is there a KSQL statement to update values in table?