Search code examples
javagradlecassandraguavaapache-beam

No repackaged dependencies when building Apache Beam Cassandra JAR

Trying to compile and use the snapshot for Apache Beam Cassandra JAR. Seems like the build does not pack the Guava dependencies within the JAR. This causes compilation to fail when the JAR is used by other code - see following Exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/beam/vendor/guava/v20_0/com/google/common/base/Preconditions
    at org.apache.beam.sdk.io.cassandra.CassandraIO$Read.withHosts(CassandraIO.java:180)
    at org.apache.beam.examples.JoinFromCassandraToCassandra.main(JoinFromCassandraToCassandra.java:26)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 2 more

I couldn't find anyway to make the gradle build package the required dependencies within the JAR.

Building using command:

gradle -p ./sdks/java/io/cassandra shadowJar

Anyone knows how this can be done?

UPDATE: So apparently a few months ago the way Beam uses Guava has changed. github.com/apache/beam/pull/7494 - So when I build the latest locally it can't find org.apache.beam.vendor.guava.v20_0.com.google.common imported by org.apache.beam.sdk.options.PipelineOptionsFactory

Solution

  • So it appears you need to add the following dependencies to make it works at the latest version. Hope it helps someone.

    compile group: 'org.apache.beam', name: 'beam-vendor-guava-20_0', version: '0.1' compile group: 'org.apache.beam', name: 'beam-vendor-grpc-1_13_1', version: '0.2'