Search code examples
springapache-kafkaspring-integrationmessagingspring-cloud-stream

Simple domain class-based Spring Kafka integration

I'm building a set of microservices in the Spring Boot framework, each of them integrating with Kafka for messaging. There appears to be 3 separate but related Spring libraries offering Kafka integration:

My goal is to abstract away the details of the underlying messaging system and provide a simple messaging service layer to my microservices to send and receive messages. I would like this service layer to work with my domain classes (POJOs) and not have the microservices being concerned with building Message instances. For example:

public interface MyMessagingService {

    void send(MyPojo obj);

    MyPojo receive();
}

Secondly, I would like to add Avro support, but first I will get it working with JSON.

To cut to the chase, there seems to be multiple ways of achieving this, which is very confusing, especially with the various Spring libraries available. What is the most straightforward way I can provide such a shared messaging layer to my microservices, where they only have to be concerned with domain classes?

I've come across @MessagingGateway from Spring Integration which looked promising, but this seems to tie in to send and reply semantics, and my services won't be expecting a reply message from Kafka.

The examples I have looked at, some linked below, still seem to have to construct Message instances themselves. Is there a simpler way of doing this?

Solution

  • If your ". . goal is to abstract away the details of the underlying messaging system and provide a simple messaging service layer to my microservices to send and receive messages", then why not just use spring-cloud-stream?

    The code developer doesn't even have to know that the code he/she writes will be part of some message system. For example,

    @SpringBootApplication
public class SampleStreamApplication  {

    public static void main(String[] args) throws Exception {
        SpringApplication.run(SampleStreamApplication.class);
    }

    @Bean
    public Function<String, String> uppercase() {
        return value -> value.toUpperCase();
    }
}

    The above is a complete and fully functioning spring cloud stream application that (in the context of Kafka binder) will receive from "input" topic and send to "output" topic the value that was passed through uppercase(..) function.

    Yes the type conversion is transparently handled for you for both JSON, Avro etc.

    Obviously there are some details, but we can certainly discuss them when you have a more concrete questions. Fo now I would suggest going through some reference documentation first.