Search code examples
spring-bootapache-kafkakafka-consumer-apispring-kafkajsonserializer

SprintBoot kafka value-serializer


I have a SpringBoot project with apache kafka ( an open-source stream-processing software ) I have this listener

@KafkaListener(topics = "test")
public String consume(Hostel hostel) throws IOException {
}

this serializer

public class HostelSerializer implements Deserializer<Hostel> {

    private final ObjectMapper objectMapper;

    public InputRequestMessageSerializer(ObjectMapper objectMapper) {
        this.objectMapper = objectMapper;
    }

    @Override
    public void configure(Map<String, ?> map, boolean b) {
        this.configure(map,b);
    }

    @SneakyThrows
    @Override
    public Hostel deserialize(String s, byte[] bytes) {
        return objectMapper.readValue(bytes, Hostel.class);
    }

    @Override
    public void close() {
        this.close();
    }
}

and in the properties:

spring:
    kafka:
        consumer:
          bootstrap-servers: localhost:9092
          group-id: group_id
          topics: test
          key-serializer: org.apache.kafka.common.serialization.StringSerializer
          value-serializer: com.kafka.config.HostelSerializer

nervertheless whe I receive a message i have this Error

Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] 
to [com.message.Hostel] for GenericMessage [.....}]

looking at the log it does not get the values from the properties:

2020-04-09 16:41:30,840 INFO  :  gid: trace= span= [main] o.a.k.c.consumer.ConsumerConfig ConsumerConfig values: 
    key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
    value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer

Solution

  • You mix de/serialization. Since you configure the consumer, you need to use only proper deserialization interfaces and implementations:

    kafka:
    consumer:
      bootstrap-servers: localhost:9092
      group-id: group_id
      topics: test
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: com.kafka.config.HostelDeserializer
    

    ... and ...

    public class HostelDeserializer implements Deserializer<Hostel> { .. }