Search code examples
apache-kafkakafka-consumer-apikafka-producer-api

Kafka serialization difference between LongSerializer and Serdes.LongSerde


We are writing a new application which publish to Kafka and we need to serialize the key of the messages as a long value. When checking the Kafka docs for serialization, seems there are two long serializer classes as LongSerializer and Serdes.LongSerde. We tried searching for a reference for the difference between the two but could not find any link explaining the difference. Appreciate if someone can please let us know / share a link which explains the difference between these. Or are these the same?

Primary link to Serializer Doc: https://kafka.apache.org/11/javadoc/org/apache/kafka/common/serialization/package-frame.html

LongSerializer: https://kafka.apache.org/11/javadoc/org/apache/kafka/common/serialization/LongSerializer.html

Serdes.LongSerde: https://kafka.apache.org/11/javadoc/org/apache/kafka/common/serialization/Serdes.LongSerde.html

Thanks.


Solution

  • As you should know, for Kafka (Brokers) messages are arrays of bytes (keys, values).

    KafkaProducer, KafkaConsumer and KafkaStreams need to know how to write and read messages - transform them from POJO to array of bytes and vice versa.

    For that purpose org.apache.kafka.common.serialization.Serializer and org.apache.kafka.common.serialization.Deserializer are used. KafkaProducer uses Serializer - to transform Key and Value to array of bytes, and KafkaConsumer uses Deserializer to transform array of bytes to Key and Value. KafkaStreams applications does both action writes, reads (to/from topic) and for that org.apache.kafka.common.serialization.Serdes are - It is some kind of wrapper for Serializer and Deserializer.

    In your example:

    • LongSerializer is a class, that should be used to translate Long to array of bytes
    • LongSerde is a class, that should be used in Kafka Streams application to read and write Long (under the hood it uses LongSerializer and LongDeserializer)

    Additional reading: