Search code examples
loggingapache-kafkaconfluent-platformlibrdkafka

What does librdkafka.redist's logs mean


This is an extract from a console log relating to Confluent.Kafka's librdkafka.redist dependency.

%3|1602097315.970|FAIL|rdkafka#consumer-2| [thrd:kfkqaapq0002d.ch.me.com:9092/bootstrap]: kfkqaapq0002d.ch.me.com:9092/bootstrap: Failed to resolve 'kfkqaapq0002d.ch.me.com:9092': No such host is known.  (after 42ms in state CONNECT)
%3|1602097315.970|FAIL|rdkafka#consumer-3| [thrd:kfkqaapq0002d.ch.me.com:9092/bootstrap]: kfkqaapq0002d.ch.me.com:9092/bootstrap: Failed to resolve 'kfkqaapq0002d.ch.me.com:9092': No such host is known.  (after 41ms in state CONNECT)
%3|1602097315.972|FAIL|rdkafka#producer-1| [thrd:kfkqaapq0003d.ch.me.com:9092/bootstrap]: kfkqaapq0003d.ch.me.com:9092/bootstrap: Failed to resolve 'kfkqaapq0003d.ch.me.com:9092': No such host is known.  (after 48ms in state CONNECT)
%3|1602097315.973|ERROR|rdkafka#producer-1| [thrd:app]: rdkafka#producer-1: kfkqaapq0003d.ch.me.com:9092/bootstrap: Failed to resolve 'kfkqaapq0003d.ch.me.com:9092': No such host is known.  (after 48ms in state CONNECT)
%3|1602097316.459|FAIL|rdkafka#producer-1| [thrd:kfkqaapq0001d.ch.me.com:9092/bootstrap]: kfkqaapq0001d.ch.me.com:9092/bootstrap: Failed to resolve 'kfkqaapq0001d.ch.me.com:9092': No such host is known.  (after 42ms in state CONNECT)

I'm trying to understand what each pipe delimited section is. Can anybody tell me? I can't find any information on it. I assume the 2nd column is some sort of timestamp, the 3rd is a status and the 4th is an error message.


Solution

  • Columns:

    • %3 - syslog severity level. Lower is more severe.
    • 1602097315.970 - seconds since epoch.
    • rdkafka#consumer-2 - client instance name, which is a combination of the (possibly configured) client.id, the client type (consumer) and a running number that is increased (in the current process) for each new client instance.
    • [thrd:kfkqaapq0002d.ch.me.com:9092/bootstrap] - what thread the log is emitted from. Application threads will be named thrd:app, all other thread names are librdkafka internal threads.
    • kfkqaapq0002d.ch.me.com:9092/bootstrap - the broker the log message corresponds to.
    • Failed to resolve 'kfkqaapq0002d.ch.me.com:9092': No such host is known. (after 42ms in state CONNECT) - the log message itself.