Search code examples
scalaapache-sparkgremlinjanusgraph

Build sbt for spark with janusgraph and gremlin scala

I was trying to setup a IntelliJ build for spark with janusgraph using gremlin scala but I am running into errors.

My build.sbt file is:

version := "1.0"

scalaVersion := "2.11.11"

libraryDependencies += "com.michaelpollmeier" % "gremlin-scala" % "2.3.0"
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.2.1"
// https://mvnrepository.com/artifact/org.apache.spark/spark-sql
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.2.1"
// https://mvnrepository.com/artifact/org.apache.spark/spark-mllib
libraryDependencies += "org.apache.spark" %% "spark-mllib" % "2.2.1"
// https://mvnrepository.com/artifact/org.apache.spark/spark-hive
libraryDependencies += "org.apache.spark" %% "spark-hive" % "2.2.1"
// https://mvnrepository.com/artifact/org.janusgraph/janusgraph-core
libraryDependencies += "org.janusgraph" % "janusgraph-core" % "0.2.0"

libraryDependencies ++= Seq(
  "ch.qos.logback" % "logback-classic" % "1.2.3" % Test,
  "org.scalatest" %% "scalatest" % "3.0.3" % Test
)

resolvers ++= Seq(
  Resolver.mavenLocal,
  "Sonatype OSS" at "https://oss.sonatype.org/content/repositories/public"
)

But I am getting errors when I try to compile code that uses gremlin scala libraries or io.Source libraries. Can someone share their build file or tell what I should modify to fix it. Thanks in advance.

So, I was trying to compile this code:

import gremlin.scala._
import org.apache.commons.configuration.BaseConfiguration
import org.janusgraph.core.JanusGraphFactory


class Test1() {
  val conf = new BaseConfiguration()
  conf.setProperty("storage.backend", "inmemory")
  val gr = JanusGraphFactory.open(conf)
  val graph = gr.asScala()
  graph.close

}

object Test{
  def main(args: Array[String]) {
    val t = new Test1()
    println("in Main")
  }
}

The errors I get are:

Error:(1, 8) not found: object gremlin import gremlin.scala._

Error:(10, 18) value asScala is not a member of org.janusgraph.core.JanusGraph val graph = gr.asScala()

Solution

  • If you go to the Gremlin-Scala GitHub page you'll see that the current version is "3.3.1.1" and that

    Typically you just need to add a dependency on "com.michaelpollmeier" %% "gremlin-scala" % "SOME_VERSION" and one for the graph db of your choice to your build.sbt (this readme assumes tinkergraph). The latest version is displayed at the top of this readme in the maven badge.

    It is not a surprise that the APi has changed when the major version of the library is different. If I change your first dependency as

    //libraryDependencies += "com.michaelpollmeier" % "gremlin-scala" % "2.3.0" //old!
libraryDependencies += "com.michaelpollmeier" %% "gremlin-scala" % "3.3.1.1"

    then your example code compiles for me.