Search code examples
scalaintellij-ideaapache-sparksbtsbt-assembly

SBT assembly falis

I am running a spark job through intellij. Job executes and gives me output. i need to take this job as jar file to server and run, but when i try to do sbt assembly it throws below error:

 [error] Not a valid command: assembly
 [error] Not a valid project ID: assembly
 [error] Expected ':' (if selecting a configuration)
 [error] Not a valid key: assembly
 [error] assembly

my sbt version is 0.13.8

below is my build.sbt file:

import sbt._, Keys._
name := "mobilewalla"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" % "2.0.0",
"org.apache.spark" %% "spark-sql" % "2.0.0")

i added a file assembly.sbt under project dir. it contains:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")

what am i missing here

Solution

  • Add these lines in your build.sbt

    assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case x => MergeStrategy.first
}
mainClass in assembly := Some("com.SparkMain")
resolvers += "spray repo" at "http://repo.spray.io"
assemblyJarName in assembly := "streaming-api.jar"

    and include these lines in your plugins.sbt file

    addSbtPlugin("io.spray" % "sbt-revolver" % "0.7.2")

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0")