Search code examples
mongodbscalaapache-sparkintellij-ideaintellij-scala

Trying to integrate mongoDB and spark, keep having errors related to "could not find or load class"


So I've been trying to integrate mongoDB and Spark, after handling reading about all the dependencies, I'm running:

  • Jdk 1.8
  • scala-sdk-2.11-7
  • Spark version 2.0.2

This is how my build.sbt file looks:

scalaVersion := "2.11.7"
libraryDependencies ++= Seq(
  "org.mongodb.spark" %% "mongo-spark-connector" % "2.0.0",
  "org.apache.spark" %% "spark-core" % "2.0.0",
  "org.apache.spark" %% "spark-sql" % "2.0.0"
)

Every time I run this simple script:

import com.mongodb.spark._
import org.apache.log4j._



object myMain {
  def main(args: Array[String]): Unit = {
    println("lol")
  }
}

It says "could not find or load main class myMain".


All I'm trying to see is if the dependencies are fine and if the script will import the libraries and work. Please help, I've been reading about this for at least a day and can not find any concrete tips except "compatibility problems".

newProject - > src -> project (src-build), build.sbt, myMain$

This is how my project tree in IntelliJ looks, in external libraries I have all libs downloaded by build.sbt file and the scala and java dependencies like I wrote above.
Thank you!


Solution

  • The answer was to change from IntelliJ ide to Eclipse.