I have an oozie job running with yarn scheduler.
Two actions in my oozie job are a spark job followed by a java job.
I have jackson version 2.6 in my spark assmbly jar and jackson 2.9 in my java assembly jar. Now my spark fails with exception
> java.lang.Exception: Incompatible Jackson version: 2.9.3
> at com.sps.analytics.batch.bi.BatchApp.executeQueries(BatchApp.scala:73)
> at com.sps.analytics.batch.bi.BatchApp$.main(BatchApp.scala:32)
How Do I force my spark application jar to use 2.6 version
// https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.6.5" force()
// https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-annotations" % "2.6.5" force()
// https://mvnrepository.com/artifact/com.fasterxml.jackson.module/jackson-module-scala
dependencyOverrides += "com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.6.5" force()
// https://mvnrepository.com/artifact/com.fasterxml.jackson.dataformat/jackson-dataformat-yaml
libraryDependencies += "com.fasterxml.jackson.dataformat" % "jackson-dataformat-yaml" % "2.6.5" force()
this is what i have in my spark build.sbt
Making the Java action a sub-workflow helps resolve the issues