Including kafka maven bound to result in convergence errors

Question

I'm developing a java-based app which produces/consumes kafka messages. In my pom I have:

    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka_2.9.2</artifactId>
        <version>0.8.2.2</version>
    </dependency>

I'm also using a maven enforcer plugin. In eclipse, which does not appear to be invoking the enforcer plugin, the program builds and runs fine. Outside eclipse, when I do 'mvn clean package' the enforcer throws a convergence error:

Dependency convergence error for org.apache.zookeeper:zookeeper:3.4.6 paths to dependency are:
+-com.mystuff:1.0
  +-org.apache.kafka:kafka_2.9.2:0.8.2.2
    +-org.apache.zookeeper:zookeeper:3.4.6
and
+-com.mystuff:1.0
  +-org.apache.kafka:kafka_2.9.2:0.8.2.2
    +-com.101tec:zkclient:0.3
      +-org.apache.zookeeper:zookeeper:3.3.1

What's the recommended way to deal with this? If course, I could exclude the transitive zookeeper dependency and add it explicitly but seems strange that I would not be able to build a kafka project without resorting to this. Typically you'd exclude the transitive dependencies when they come in different versions from different top-level dependencies. Here - I'm having a problem with one and only one top-level dependency, kafka.

Answer 1

Kafka build clearly doesn't run with an enforcer, so we accidentally pull in the same dependency twice with different versions.

My suggestion:

1. Use Exclude to resolve this in your own project
2. Open a ticket for Apache Kafka (issues.apache.org/jira/browse/KAFKA) and explain the problem. Perhaps you can even contribute a pull request that fixes the issue (http://kafka.apache.org/contributing.html)