Search code examples
apache-sparkapache-zeppelin

Zeppelin - Spark 1.6 issues with dependencies

Recently, dependencies have migrated from worksheet to interpreter as per below screenshot: enter image description here

Unfortunately, when applying dependencies as per below: enter image description here

The error is produced when calling spark context: enter image description here

or another one as java.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class

does anyone know how to fix it? I removed jackson libraries without any result:

rm -f zeppelin-server/target/lib/jackson-*
rm -f zeppelin-zengine/target/lib/jackson-*

and tried to upload com.fasterxml.jackson.core into artifact in dependencies but this does not exist in https://repo1.maven.org/maven2/

Solution

  • I was able to input dependencies into /usr/local/zeppelin/conf/zeppelin-env.sh

    export JAVA_HOME=/usr

    export MASTER=local[*]

    export ZEPPELIN_PORT=8081

    export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.2.0"

    which works fine