Search code examples
javaapache-kafkalog4japache-kafka-streams

Kafka Streams ignores log4j.properties file


I am trying to change logging style in Kafka Streams 2.7.1 and do not see any reaction. Project structure: enter image description here

Content of log4j.properties:

log4j.rootLogger=DEBUG, console

log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=---------%d{HH:mm:ss,SSS} %-5p %-60c %x - %m%n

In pom.xml I have added next deps:

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.5</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.6.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.11.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.11.0</version>
        </dependency>

But log messages of the application do not match with pattern (no dashes in the beginning):

654 [kafka-producer-network-thread | streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
654 [kafka-producer-network-thread | streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
729 [streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-consumer, groupId=streams-wordcount] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
729 [streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1] WARN org.apache.kafka.clients.NetworkClient - [Consumer clientId=streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-consumer, groupId=streams-wordcount] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected
741 [streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1] INFO org.apache.kafka.streams.processor.internals.StreamThread - stream-thread [streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1] Processed 0 total records, ran 0 punctuators, and committed 0 total tasks since the last update
754 [kafka-producer-network-thread | streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] Connection to node -1 (localhost/127.0.0.1:9092) could not be established. Broker may not be available.
754 [kafka-producer-network-thread | streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=streams-wordcount-87c688dc-7ade-4b4b-bd06-1187ffa4755a-StreamThread-1-producer] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

Why does it not work?


Solution

  • As @OneCricketeer mentioned in comments - I had to fix some mistakes with dependencies. And I had to add to replace slf4j with slf4j-log4j12 for ability to use log4j.property file. As final solution I have this dependencies in pom.xml:

        <dependencies>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-streams</artifactId>
            <version>2.7.1</version>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.7.25</version>
        </dependency>
    </dependencies>