Search code examples
apache-kafkacompressionkafka-consumer-apikafka-producer-api

Kafka compression fail in comsumer side


I have enabled snappy compression on the producer side with the following change:

props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG,"snappy");

Now if any change is required on consumer side for this? I checked the Consumer.config but did not find any compression_type parameter there.

And When I ran that code messages are not coming to consumer side and I am getting the following error:

[1/10/20 17:25:39:759 UTC] 00018027 org.apache.kafka.clients.consumer.KafkaConsumer              I [Consumer clientId=com.ora.def.lieo, groupId=group-cedm] Seeking to offset 320 for partition Party-Resolved-Check-0
[1/10/20 17:25:49:760 UTC] 00018029 com.ibm.cedm.kafka.KafkaConsumerService                      E Received exception when fetching the next record from Party-Resolved-Check-0. If needed, please seek past the record to continue consumption.

Solution

  • There is no decompression config required. Consumers automatically handle this.

    It's not clear how your error is related to compression