Search code examples
log4japache-beamlog4j2slf4j

How can I use Log4j2 logging on BEAM workers?


I'm trying to use Log4j2 logging implementation together with Slf4j api in the BEAM pipeline. In maven it looks like this:

    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-core</artifactId>
      <version>2.20.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.logging.log4j</groupId>
      <artifactId>log4j-slf4j2-impl</artifactId>
      <version>2.20.0</version>
    </dependency>

It works fine locally. Also the logging works fine for the container where I start the pipeline. However, inside the workers this setup causes some issues:

Caused by: org.apache.logging.log4j.LoggingException: log4j-slf4j2-impl cannot be present with log4j-to-slf4j

This additional library log4j-to-slf4j is not part of my project and it's not listed in dependencies (also not in transitive dependencies). Looks like it's added by BEAM sdk itself. The question is: how can I configure BEAM to use my version of Log4j?


Solution

  • So the solution is easy. Looks like BEAM uses slf4j v1 libs so these dependencies work fine with it and use log4j2 as a backend for logging:

        <dependency>
          <groupId>org.apache.logging.log4j</groupId>
          <artifactId>log4j-core</artifactId>
          <version>${log4j.version}</version>
        </dependency>
        <dependency>
          <groupId>org.apache.logging.log4j</groupId>
          <artifactId>log4j-slf4j-impl</artifactId>
          <version>${log4j.version}</version>
        </dependency>