Search code examples
apache-kafkaapache-kafka-streamsspring-cloud-streamspring-cloud-stream-binder-kafka

Will GlobalKTable always have only 1 record per partition key even if topic compaction for records has not kicked in?

I have a Kafka topic that has compaction enabled by setting cleanup.policy=compact. My segment.bytes property is set to a slightly larger values (100Mb) so that my brokers are performing well.

If I have a Kafka streams application that is using the topic as GlobalKTable and there are multiple records for a single partition key in the topic, will the application receive only 1 record in the GlobalKTable or it will have two till the compaction kicks in?

Solution

  • There will be only 1 record in the GlobalKTable, as Tables are by default RocksDB state stores which is a key-value store. Hence even though the topic has more than one for a key, the table will have only one.