Search code examples
apache-kafkasuppress-warningskafka-consumer-api

Don't print the kafka-console-consumer warnings

I am trying to debug encrypted messages onto a Kafka cluster. Obviously these messages are full of non printable characters and are not usable on a console, so I wanted to save the output in a file like this:

kafka-console-consumer \
    --zookeeper 0.zookeeper.local,1.zookeeper.local \
    --max-messages 1 \
    --topic MYTOPIC > /tmp/message

I am unable to decrypt the resulting message, because the output contains, along with the ciphertext, warning messages such as:

[2016-02-24 11:52:47,488] WARN Reconnect due to socket error: null (kafka.consumer.SimpleConsumer)

Is there a way to get one single message in a file and NOT embed the warnings in that file?

Solution

  • There is a special logger config file for inbound console tools. Please, change the logger level from WARN to OFF in [kafka_home]/config/tools-log4j.properties file. The file should look like:

    tools-log4j.properties

    log4j.rootLogger=OFF, stdout 

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c)%n

    After it you will get only messages in console.