Search code examples
scalaapache-sparkscala-2.11

Run Spark in standalone mode with Scala 2.11?

I follow the instructions to build Spark with Scala 2.11:

mvn -Dscala-2.11 -DskipTests clean package

Then I launch per instructions:

./sbin/start-master.sh

It fails with two lines in the log file:

Failed to find Spark assembly in /etc/spark-1.2.1/assembly/target/scala-2.10
You need to build Spark before running this program.

Obviously, it's looking for a scala-2.10 build, but I did a scala-2.11 build. I tried the obvious -Dscala-2.11 flag, but that didn't change anything. The docs don't mention anything about how to run in standalone mode with scala 2.11.

Thanks in advance!

Solution

  • Before building you must run the script under:

    dev/change-version-to-2.11.sh

    Which should replace references to 2.10 with 2.11.

    Note that this will not necessarily work as intended with non-GNU sed (e.g. OS X)