I have a Bluemix Node.js (6.1.0) application that uses node-rdkafka 1.0.3. It seems to be working ok but there are tons of error events like Error: Local: Broker Transport Failure
or Error: Local: Authentication failure
.
The producer options I have set are:
var producer_opts = {
"metadata.broker.list":env.messagehub.brokers,
"security.protocol":"sasl_ssl",
"ssl.ca.location":env.messagehub.calocation,
"sasl.mechanisms":"PLAIN",
"sasl.username":env.messagehub.user,
"sasl.password":env.messagehub.password,
"api.version.request":true,
"socket.timeout.ms": 10000,
"dr_msg_cb":true
};
Consumer has similar settings plus the group.id
tag.
I wonder if I should be worrying for theese errors and if there is a way to eliminate them. Thanks!
You are probably hitting https://github.com/edenhill/librdkafka/issues/1218.
In many cases, as you've noticed, these errors are harmless. The library node-rdkafka is based onto, librdkafka, always connects to all brokers in the cluster. Brokers your applications doesn't interact with will close the idle connections after a while leading to these error messages in your clients.
Unfortunately we don't have a recommended way to eliminate them at the moment. We are currently working on a potential solution to at least reduce their rate and maybe get rid of them.
Update:
With the most recent releases of node-rdkafka (>2.2), you can get rid of all the noisy logs by setting the following properties when creating clients:
'broker.version.fallback': '0.10.2.1',
'log.connection.close' : false