java.nio.BufferUnderflowException when trying to connect to kafka broker

Question

I am seeing the following error message when I try to connect to my Kafka broker:

2020-08-10 15:59:54.974 ERROR 31447 --- [   scheduling-1] c.m.f.adapter.KafkaMessageListener       : Failed to consume message

java.lang.IllegalStateException: There are no in-flight requests for node -1
    at org.apache.kafka.clients.InFlightRequests.requestQueue(InFlightRequests.java:62) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.InFlightRequests.completeNext(InFlightRequests.java:70) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:838) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:558) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:236) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:463) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1275) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1241) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) ~[kafka-clients-2.5.0.jar:na]
    at com.mastercontrol.formconfiguration.adapter.KafkaMessageListener.listen(KafkaMessageListener.java:38) ~[main/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

2020-08-10 15:59:55.560 ERROR 31447 --- [   scheduling-1] c.m.f.adapter.KafkaMessageListener       : Failed to consume message

java.nio.BufferUnderflowException: null
    at java.base/java.nio.Buffer.nextGetIndex(Buffer.java:647) ~[na:na]
    at java.base/java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:400) ~[na:na]
    at org.apache.kafka.common.protocol.ByteBufferAccessor.readInt(ByteBufferAccessor.java:43) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.common.message.ResponseHeaderData.read(ResponseHeaderData.java:93) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.common.message.ResponseHeaderData.<init>(ResponseHeaderData.java:65) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.common.requests.ResponseHeader.parse(ResponseHeader.java:66) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.parseStructMaybeUpdateThrottleTimeMetrics(NetworkClient.java:722) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.handleCompletedReceives(NetworkClient.java:839) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:558) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1308) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1248) ~[kafka-clients-2.5.0.jar:na]
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) ~[kafka-clients-2.5.0.jar:na]
    at com.mastercontrol.formconfiguration.adapter.KafkaMessageListener.listen(KafkaMessageListener.java:38) ~[main/:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
    at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE]
    at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.2.7.RELEASE.jar:5.2.7.RELEASE]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
    at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

The weird thing is, I am only seeing this on my native Mac. When I run the same code in docker on an alpine system, the client connects just fine.

I am running confluent version 5.5.0 and kafka client 2.5.0. I am running Zulu 11 JDK on both my Mac and in my container. I am on the latest MacOS version.

I am absolutely stumped. Can anyone help me?

Answer 1

Ok, I fixed the issue. It turns out, in my case, that I was running unbound. Im not sure if It was a mis-configuration or not, but uninstalling unbound solved my issue.