Search code examples
javaapache-kafkakafka-consumer-apiapache-kafka-streams

Can I use different set of kafka brokers for storing the state of Kafka streams application?

I just want to know if its possible to use a different set of brokers for storing the kafka streams state topics. And also, is it possible to use another storage system, like amazon s3 as kafka streams sink with exactly once semantics? I don't want to first use an output topic, then consume from it and store in s3.

Thanks!

Solution

  • Kafka Streams can only use one cluster for its data

    You'd need to use Kafka Connect to write to S3 prior to KIP-405