Helidon MP: How to send slf4j log messages to Kafka broker?

Question

I am trying to send slf4j log messages in my Helidon MP application to a Kafka server that runs on port 9092. I have the following class as an example:

import lombok.extern.slf4j.Slf4j;

@Slf4j
public class Service {

  private final ConfigProvider configProvider;

  @Inject
  public Service(ConfigProvider configProvider) {
    this.configProvider = configProvider;
  }

  public String getString() {
    String msg = String.format("%s %s !", configProvider.getString());
    log.info("Entered getString() method");
    return msg;
  }
}

I also have a logging.xml file which specifies the Appender as KafkaAppender:

<Configuration>
    <Appenders>
        <Kafka name="KafkaAppender" topic="app-logs"
               syncSend="false">
            <Property name="bootstrap.servers"
                      value="localhost:9092"/>
        </Kafka>
    </Appenders>
    <Loggers>
        <Logger name="org.apache.kafka" level="WARN"/> <!-- avoid recursive logging -->
        <Root level="INFO">
            <AppenderRef ref="KafkaAppender"/>
        </Root>
    </Loggers>
</Configuration>

However, when I run the application, I get the following errors:

2022-11-28 14:23:17,358 main ERROR No layout provided for KafkaAppender
2022-11-28 14:23:17,362 main ERROR Null object returned for Kafka in Appenders.
2022-11-28 14:23:17,364 main ERROR Unable to locate appender "KafkaAppender" for logger config "root"

Any suggestions on how to make KafkaAppender work with Helidon?

Answer 1

Helidon does nothing special for logging. The only thing to note is that Helidon uses JUL (java.util.logging) for logging.

KafkaAppender is a Log4J2 appender, however you said "specify slf4j to send message to Kafka".

1. Setup the SLF4J to JUL bridge

See the javadocs.

Add the following dependency to your pom.xml:

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>jul-to-slf4j</artifactId>
</dependency>

You can setup the bridge either programmatically or by configuration.

Programmatically

import org.slf4j.bridge.SLF4JBridgeHandler;

SLF4JBridgeHandler.removeHandlersForRootLogger();
SLF4JBridgeHandler.install();

Using logging.properties

If you are using Helidon SE, you need to have code in your main class to load logging.properties. Helidon provides a utility for that, see below. If you are using Helidon MP, this is done for you automatically.

import io.helidon.common.LogConfig;

LogConfig.configureRuntime();

If you already have the statement above in your main method, you can setup the bridge by adding the following to your logging.properties file.

handlers = org.slf4j.bridge.SLF4JBridgeHandler

If you don't have a logging.properties file yet, create it under src/main/resources so that it is added to your project JAR.

2. Setup the SLF4J implementation

Import the Log4J2 bom in your pom.xml:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-bom</artifactId>
            <version>2.19.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Add the following dependencies to your pom.xml:

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j-impl</artifactId>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
</dependency>

See the docs.

3. Configure Log4J2

See the docs.

The default path for a XML configuration file is log4j2.xml. Create the file at src/main/resources/log4j2.xml

<Configuration>
    <Appenders>
        <Kafka name="KafkaAppender" topic="app-logs" syncSend="false">
            <PatternLayout pattern="%date %message" />
            <Property name="bootstrap.servers" value="localhost:9092"/>
        </Kafka>
    </Appenders>
    <Loggers>
        <Logger name="org.apache.kafka" level="WARN"/> <!-- avoid recursive logging -->
        <Root level="INFO">
            <AppenderRef ref="KafkaAppender"/>
        </Root>
    </Loggers>
</Configuration>

Add the kafka-clients dependency to your pom.xml:

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
</dependency>

---

Please note that there nothing specific to Helidon here apart from dependency version management and the utility provided to load logging.properties. You can use the same steps on any "plain" Java project.