Search code examples
apache-kafkaspring-kafka

Kafka does not release the thread, if it does not find topic


I noticed that if there are no topic that producer produces messages in, because kafka does not release the thread we can not get response to my http request.

Is there a way to configure Kafka that if we get UNKNOWN_TOPIC_OR_PARTITION ignore it and do not try to send message.

I see that the below error is being stacked.

 [Producer clientId=session-service-1] Error while fetching metadata with correlation id 28 : {*****.session-service.test.v1=UNKNOWN_TOPIC_OR_PARTITION}

Although I catch any exceptions it breaks the flow.

try {
            ListenableFuture<SendResult<String, SessionEvent>> future = kafkaTemplate.send(topicName, sessionEvent);
            future.addCallback(sessionEvent.getFutureCallback());
        } catch (Throwable e) {
            log.error("Error", e);
        }

Solution

  • It turned out that although it was async method, my application was not configured for async methods.

    So it was executing the method sync. But I did not throw any exceptions.. So Kafka was still trying to send the message to the invalid topic.