Search code examples
apache-kafkaapache-nifikafka-producer-apisasl

Failed to send SSL Close message when using KafkaPublish processor to write to kafka topic

I am using KafkaPublish processor to publish messages to a kafka topic secured using SASL_SSL security protocol (we are not using Kerberos).

I get the below error: 

WARN Failed to send SSL Close message  
(org.apache.kafka.common.network.SslTransportLayer)
java.io.IOException: An existing connection was forcibly closed by the 
remote host
    at sun.nio.ch.SocketDispatcher.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:51)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
    at 
  org.apache.kafka.common.network.SslTransportLayer.flush(SslTransportLayer.java:209)
    at org.apache.kafka.common.network.SslTransportLayer.close(SslTransportLayer.java:172)
    at org.apache.kafka.common.utils.Utils.closeAll(Utils.java:718)
    at org.apache.kafka.common.network.KafkaChannel.close(KafkaChannel.java:61)
    at org.apache.kafka.common.network.Selector.doClose(Selector.java:746)
    at org.apache.kafka.common.network.Selector.close(Selector.java:734)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:532)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:424)
    at kafka.network.Processor.poll(SocketServer.scala:628)
    at kafka.network.Processor.run(SocketServer.scala:545)
    at java.lang.Thread.run(Thread.java:748)

I am able to publish into the topic from command prompt.

When i use SASL_SSL in KafkaPublish Kerberos service name is a mandatory field. But since we are not using Kerberos what value should I set for this field?

Configure Processor

When i set the dynamic properties sasl.mechanism and sasl.jaas.configuration I get invalid configuration parameter warning.

Am I missing any configurations? Any help will be appreciated. Thanks in advance.

Solution

  • NiFi's PublishKafka only support SASL with mechanisms other than Kerberos since version 0_11.

    So you want to be sure you use PublishKafka_0_11 or PublishKafka_1_0 if you need to use SASL Plain for example.