Search code examples
c#apache-kafkakafka-producer-api.net-4.7.2

Building a Kafka producer immediately results "Disconnected while requesting ApiVersion" error

I downloaded kafka_2.13-2.5.1, extracted it into a folder, edited producer.properties, to enable Snappy compression, and left the rest vanilla. I start whatever it is (server? cluster? hell knows) with the command

bin/windows/zookeeper-server-start.bat config/zookeeper.properties

It starts and shows that it is listening on port 2181:

INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)

In my C# code, I create an error handler:

void KafkaErrorHandler(IProducer<Null, string> producer, Error error)
{
    Console.WriteLine(error.Reason);
}

Then I am trying to build a producer:

kafkaProducerConfig = new ProducerConfig
{
    CompressionType = CompressionType.Snappy,
    BootstrapServers = "localhost:2181", // Docs said 9092 plain, 9093 SSL. Huh?
    ClientId = Dns.GetHostEntry(Environment.MachineName).HostName
};

var pb = new ProducerBuilder<Null, string>(kafkaProducerConfig);
pb.SetErrorHandler(KafkaErrorHandler);
kafkaProducer = pb.Build();

As soon as the code reaches the last line, it instantaneously enters the error handler, and the Error object shows this:

{localhost:2181/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 1ms in state APIVERSION_QUERY)}

The Kafka console window immediately begins to rain endless warnings, until I kill the application:

[2020-12-01 15:53:39,486] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:53:39,781] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:53:40,093] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:53:41,409] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:53:45,683] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:53:55,686] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:54:05,688] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:54:13,683] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:54:23,682] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:54:33,573] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:54:43,579] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:54:53,583] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:55:03,499] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:55:13,504] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:55:23,501] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:55:33,505] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:17,879] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:18,175] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:18,494] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:19,809] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:24,083] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:34,087] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:44,087] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:56:52,080] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:57:02,079] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:57:11,968] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:57:59,379] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:57:59,675] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:57:59,986] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:01,302] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:05,575] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:15,580] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:25,579] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:33,575] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:43,575] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:58:53,465] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:59:03,465] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:59:13,468] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:59:23,384] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:59:33,387] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:59:43,387] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 15:59:53,393] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:03,391] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:13,399] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:22,206] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:32,210] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:39,848] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:49,851] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:00:59,851] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:01:09,856] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:01:18,576] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:01:28,580] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:01:38,580] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:01:46,192] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:01:55,227] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:05,231] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:14,033] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:21,884] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:29,387] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:39,399] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:50,368] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:53,368] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:53,467] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:53,895] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:54,612] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)
[2020-12-01 16:02:58,736] WARN Exception causing close of session 0x0: Unreasonable length = 1718383973 (org.apache.zookeeper.server.NIOServerCnxn)

If I add SecurityProtocol = SecurityProtocol.Ssl, to producer config, then I get the following error instead:

{ssl://localhost:2181/bootstrap: SSL handshake failed: Disconnected: connecting to a PLAINTEXT broker listener? (after 2ms in state CONNECT)}

What else does it need, to make up its mind and start working?

Solution

  • I start whatever it is with zookeeper-server-start

    The Kafka console window

    That's a Zookeeper Server, not Kafka

    You need a separate window that starts kafka-server-start with the config/server.properties file.

    Then your code would use port 9092 (by default) as you seem to have commented on in your shown code.

    If I add SecurityProtocol = SecurityProtocol.Ssl, to producer config, then I get the following error instead

    From what I can tell, you never setup any SSL properties in the Kafka server, so that wouldn't change anything