Search code examples
scalaapache-sparkcassandrasbtspark-cassandra-connector

sbt package not adding dependencies


I am trying to build jar using sbt package.

build.sbt:

name := "Simple Project"
version := "0.1"

scalaVersion := "2.11.8"
val sparkVersion = "2.3.2"
val connectorVersion = "2.3.0"
val cassandraVersion = "3.11"

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-sql" % sparkVersion % "provided",
  "org.apache.spark" %% "spark-hive" % sparkVersion % "provided",
  "org.scalaj" %% "scalaj-http" % "2.4.2",
  "com.datastax.spark" %% "spark-cassandra-connector" % connectorVersion
)

The sbt package runs successfully but does not add spark-cassandra-connector and scalaj-http to the final jar created.

Do I need to add anything?


Solution

  • If you want the jar to contain all your dependencies, you have to use the sbt assemlbly plugin:

    https://github.com/sbt/sbt-assembly