Search code examples
javaapache-kafkajava-8apache-kafka-streams

Writing KStream<String, Map<String, Long>> to a Kafka Topic

I am reading an from a topic in Kafka and using a streams application to apply changes to my data.

Finally I have a stream looking like this KStream<String, Map<String, Long>>. The key (String) is a hash, the value a Map which contains also a key (String) which is just a stemmed word and the value (Long) a count.

It would be great if somebody could point me to a nice way of writing this back to a Kafka topic. I know I cloud convert the Map into a proprietary String representation. But I think that there are more elegant solutions to this problem, e.g. using Serdes and apache avro?

Many Thanks Eduard

Solution

  • You could use Avro or any binary format

    A Map<String,?> is also represented by a JSON object, which doesn't need to be proprietary. Kafka comes with Jackson, so define an ObjectMapper and write the Map to a JSONObject instance