Search code examples
scalaapache-sparkdatastax-enterprisecassandra-2.0

Exception in thread "main" java.lang.NoClassDefFoundError: com/twitter/chill/KryoBase


I am writing a simple spark-cassandra program in java with datastax cassandra but getting below exception

Exception in thread "main" java.lang.NoClassDefFoundError: com/twitter/chill/KryoBase Caused by: java.lang.ClassNotFoundException: com.twitter.chill.KryoBase

pom.xml

 <dependency>
        <groupId>com.datastax.dse</groupId>
        <artifactId>dse-spark-dependencies</artifactId>
        <version>5.1.1</version>
        <exclusions>
            <exclusion>
                <groupId>com.datastax.dse</groupId>
                <artifactId>dse-java-driver-core</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.apache.solr</groupId>
                <artifactId>solr-solrj</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

App.java

 SparkConf conf = new SparkConf().setMaster("local").setAppName("SparkJavaFirst")
                    .set("spark.cassandra.connection.host", "localhost");     
JavaSparkContext sc = new JavaSparkContext(conf);  
JavaRDD<CassandraRow> cassandraJavaRDD =  CassandraJavaUtil.javaFunctions(sc).
                    cassandraTable("my_keyspace","my_table" ).
                    select("my_column");   

Program fails at JavaSparkContext sc = new JavaSparkContext(conf);

Any help will be appreciated.


Solution

  • This issue got resolved as I removed the exclusion of

     <groupId>com.datastax.dse</groupId>  
      <artifactId>dse-java-driver-core</artifactId>