java.lang.ClassCastException: com.datastax.driver.core.DefaultResultSetFuture cannot be cast to shade.com.datastax

Question

I am getting this error, when trying to run above code.

Error :

  Exception:
    Caused by: java.lang.ClassCastException: com.datastax.driver.core.DefaultResultSetFuture cannot be cast to shade.com.datastax.spark.connector.google.common.util.concurrent.ListenableFuture
        at com.datastax.spark.connector.writer.AsyncExecutor.com$datastax$spark$connector$writer$AsyncExecutor$$tryFuture$1(AsyncExecutor.scala:38) ~[spark-cassandra-connector_2.11-2.3.1.jar:2.3.1]
        at com.datastax.spark.connector.writer.AsyncExecutor.executeAsync(AsyncExecutor.scala:71) ~[spark-cassandra-connector_2.11-2.3.1.jar:2.3.1]
        at com.datastax.spark.connector.writer.TableWriter$$anonfun$writeInternal$1$$anonfun$apply$2.apply(TableWriter.scala:234) ~[spark-cassandra-connector_2.11-2.3.1.jar:2.3.1]
        at com.datastax.spark.connector.writer.TableWriter$$anonfun$writeInternal$1$$anonfun$apply$2.apply(TableWriter.scala:233) ~[spark-cassandra-connector_2.11-2.3.1.jar:2.3.1]
        at scala.collection.Iterator$class.foreach(Iterator.scala:893) ~[scala-library-2.11.8.jar:?]

Code snippets.

<dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.11</artifactId>
            <version>2.3.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.11</artifactId>
            <version>2.3.1</version>
        </dependency>
        <dependency>
            <groupId>com.datastax.spark</groupId>
            <artifactId>spark-cassandra-connector_2.11</artifactId>
            <version>2.3.1</version>
        </dependency>

        <dependency>
            <groupId>com.datastax.cassandra</groupId>
            <artifactId>cassandra-driver-mapping</artifactId>
            <version>3.2.0</version>
        </dependency>

Can you please help to resolve this issue? Thanks in advance.

Answer 1

This can only happen if a non shaded version of the Java Driver is included. Because of Guava incompatibilities we need to shade the Java Driver in the Spark Cassandra Connector. This means that any additional Java Driver based libraries will break the shading and cause reflection errors like the one above.

If you need to manually do something to the Java Driver version, use the unshaded version of the Spark Cassandra Connector, include your own libraries, then shade guava again.