Learning Scala from the Scala for Data Science book and the companion Github repo, here I am particularly talking about the build file for Chapter 2, copied below (with minor modification) for reference.
name := "Logistic_regression"
organization := "My Organisation"
version := "0.1.0-SNAPSHOT"
scalaVersion := "2.11.7" // 3.2.2 does not work
libraryDependencies ++= Seq(
"org.scalanlp" %% "breeze" % "0.11.2",
"org.scalanlp" %% "breeze-natives" % "0.11.2",
"org.slf4j" % "slf4j-simple" % "1.7.5"
)
Being new to scala from python and C++, the idea of how different scala environments, dependencies etc. work is not entirely clear to me yet, but can I think of the build.sbt as a form of Dockerfile, that can take care of the dependencies on any host by creating its own sandbox?
The repo itself uses scalaVersion := "2.11.7" which works fine. But I wanted to match it to my system scala version (see below for the version details).
della@dell-xps ~/s/l/s/chap02 (master)> scala --version
Scala code runner version 3.2.2 -- Copyright 2002-2023, LAMP/EPFL
della@dell-xps ~/s/l/s/chap02 (master)> sbt --version
sbt version in this project: 1.8.2
sbt script version: 1.8.2
But changing it to 3.2.2 results in the following stack-trace (just the top part).
[warn] Note: Unresolved dependencies path:
[error] sbt.librarymanagement.ResolveException: Error downloading org.scalanlp:breeze_3:0.11.2
[error] Not found
[error] Not found
[error] not found: /home/della/.ivy2/localorg.scalanlp/breeze_3/0.11.2/ivys/ivy.xml
[error] not found: https://repo1.maven.org/maven2/org/scalanlp/breeze_3/0.11.2/breeze_3-0.11.2.pom
So why is it that build.sbt cannot resolve the dependencies for a more recent scala version? Or, should it?
Further, if I want to work on the same project with the newer scala version, should I do it without sbt, and somehow installing the dependencies at the system level, available to all projects? How do I go around it?
can I think of the build.sbt as a form of Dockerfile, that can take care of the dependencies on any host by creating its own sandbox?
There is no sandboxing here in Docker sense, you're not creating an isolated environment (@SethTisue commented that this can be considered as sandboxing in the sense that sbt manages dependencies itself, rather than relies on the ones installed systemwide). This is using a build tool like Sbt, Mill, Fury, Cbt in Scala, Maven or Gradle in Java, Bazel in Java/C++/Go, Pants in Python/Go/Java/Scala/Kotlin, Make in C++, Pip in Python, Cabal or Stack in Haskell etc.
Scala 2 and Scala 3 are quite different
https://docs.scala-lang.org/scala3/reference/
Scala 3 is still quite new version of Scala. Not all dependencies are published for Scala 3. Not all of them can be easily migrated into Scala 3 (because of Scala 2 macros etc.).
Scala 2.10, 2.11, 2.12, 2.13 are binary incompatible. That's why we use %% for Scala libraries: "org.scalanlp" %% "breeze" % "0.11.2" is "org.scalanlp" % "breeze_2.11" % "0.11.2" while using scalaVersion := "2.11.7". But there is Scala 2.13 - Scala 3 interop
https://docs.scala-lang.org/scala3/guides/migration/compatibility-intro.html
There is compatibility mode CrossVersion.for3Use2_13/CrossVersion.for2_13Use3
https://docs.scala-lang.org/scala3/guides/migration/compatibility-classpath.html
You're using quite old versions of your dependencies. As you can see, Breeze and Breeze-natives 0.11.2 were published for Scala 2.10, 2.11. They are published for Scala 3 since Breeze 2.0
https://mvnrepository.com/artifact/org.scalanlp/breeze
https://mvnrepository.com/artifact/org.scalanlp/breeze-natives
So try the following build.sbt
scalaVersion := "3.2.2"
libraryDependencies ++= Seq(
"org.scalanlp" %% "breeze" % "2.1.0",
"org.scalanlp" %% "breeze-natives" % "2.1.0",
"org.slf4j" % "slf4j-simple" % "2.0.7"
)
For Breeze 1.x there is no Scala-3 version but there is Scala-2.13 version, so you can try compatibility mode
scalaVersion := "3.2.2"
libraryDependencies ++= Seq(
"org.scalanlp" %% "breeze" % "1.0" cross CrossVersion.for3Use2_13,
"org.scalanlp" %% "breeze-natives" % "1.0" cross CrossVersion.for3Use2_13,
"org.slf4j" % "slf4j-simple" % "2.0.7"
)
But before Breeze 1.0 there was no Scala-2.13 version, so you can use only Scala 2.10-2.12 with them
scalaVersion := "2.12.17"
libraryDependencies ++= Seq(
"org.scalanlp" %% "breeze" % "0.13.2",
"org.scalanlp" %% "breeze-natives" % "0.13.2",
"org.slf4j" % "slf4j-simple" % "2.0.7"
)
If you're using sbt, most probably you will not need to install anything in your system (except sbt and Java). Sbt will resolve dependencies itself.
If you installed something in the system, most probably this is irrelevant for sbt builds.