Search code examples
apache-sparkmaven-2slf4j

Maven SL4J multiple bindings, previous solutions fail


I have a project that has multiple SLF4J bindings. I have read and tried the solutions in this SO post, this other SO post and the slf4j website.

What I see if this when I run the code is

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/Mike/.m2/repository/org/jlab/coat/coat-libs/5.1-SNAPSHOT/coat-libs-5.1-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/Mike/.m2/repository/org/slf4j/slf4j-log4j12/1.7.16/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]

However in my pom.xml file, I already have

    <dependency>
        <groupId>org.jlab.coat</groupId>
        <artifactId>coat-libs</artifactId>
        <version>5.1-SNAPSHOT</version>
        <exclusions>
            <exclusion>
                <groupId>org.slf4j</groupId>
                <artifactId>slf4j-log4j12</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

and in the mvn dependency:tree I do not see this dependency for the exclusion jar, only for the Spark jar, i.e.

[INFO] com.IKP:DCdatabase:jar:0.0.1-SNAPSHOT
[INFO] +- org.jlab.coat:coat-libs:jar:5.1-SNAPSHOT:compile
[INFO] +- org.apache.spark:spark-core_2.11:jar:2.2.1:compile
...
...
...
[INFO] |  +- org.slf4j:slf4j-api:jar:1.7.16:compile
[INFO] |  +- org.slf4j:jul-to-slf4j:jar:1.7.16:compile
[INFO] |  +- org.slf4j:jcl-over-slf4j:jar:1.7.16:compile
[INFO] |  +- log4j:log4j:jar:1.2.17:compile
[INFO] |  +- org.slf4j:slf4j-log4j12:jar:1.7.16:compile

I have also tried a few more steps, such as clean out my .m2 directory, and build all sources from scratch, but I still see this double binding.

The nuance that occurs is that the log suppression for Spark does occur, i.e.

    Logger.getLogger("org.apache.spark.SparkContext").setLevel(Level.WARN);
    Logger.getLogger("org").setLevel(Level.OFF);
    Logger.getLogger("akka").setLevel(Level.OFF);

No longer set the levels to off and I see all the levels.

Is there another way to remove this multiple binding for SLF4J?


Solution

  • It looks to me like coat-libs has been packaged as an uber jar which is why slf4j doesn't show up as a dependency when you do mvn dependency:tree. This explains why your exclusion isn't working.

    I would recommend taking a look at the maven shade plugin for packaging your jar. Then, you can use a filter to exclude the slf4j dependencies from coat-libs using a filter.

    For example, maybe something like:

      <build>
        <plugins>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.1.1</version>
            <executions>
              <execution>
                <phase>package</phase>
                <goals>
                  <goal>shade</goal>
                </goals>
                <configuration>
                  <filters>
                    <filter>
                      <artifact>org.jlab.coat:coat-libs</artifact>
                      <excludes>
                        <exclude>org/slf4j/**</exclude>
                      </excludes>
                    </filter>
                    <filter>
                      <artifact>*:*</artifact>
                      <excludes>
                        <exclude>META-INF/*.SF</exclude>
                        <exclude>META-INF/*.DSA</exclude>
                        <exclude>META-INF/*.RSA</exclude>
                      </excludes>
                    </filter>
                  </filters>
                </configuration>
              </execution>
            </executions>
          </plugin>
        </plugins>
      </build>