Search code examples
spring-bootapache-kafkakafka-consumer-apikafka-producer-apispring-kafka

Kafka Implementation for Rest API

I have a microservice built using spring boot. The Load balancer has a fixed timeout and this service does some large calculation and thus the response gets timed out when I call it from the UI.

I figured that Spring Kafka will be very suitable in this scenario and can help me to queue my responses into kafka producer and then read them from consumer service. But the kafka topic that we create is constant for the entire service, then how can I retrieve the results back from this queue corresponding to the specific request made from the UI.(Since there will be multiple requests for the same service). Also if there is some good example for this implementation would be very helpful

Solution

  • What are you looking for are basically two things

    1. Creating a pipeline to process all your async request.

    2. Tracking System to track the state of your job.

    So you should assign a trackingId to each request that you are pushing in Kafka queue and as the message flow through the pipeline updates the state corresponding to that trackingId. In order to know the exact status, you should expose an API (which you can link with your UI) which will give the exact status of your job.