Search code examples
pythonscalaapache-sparkjupyter-notebookapache-toree

Default lib jars folder for Apache Toree kernel


Say I want a default relative lib folder in jupyter notebook project directory where I can download custom jars so that I can import later without %addjar magic.

I was under impression I can do something like:

"__TOREE_OPTS__": "--jar-dir=./lib/"

in ~/.local/share/jupyter/kernels/apache_toree_scala/kernel.json, but this doesn't work.

What am I missing?


Solution

  • The following kernel.json "just works":

    # ~/.local/share/jupyter/kernels/apache_toree_scala/kernel.json
    {
      "argv": [
        "/home/sergey/.local/share/jupyter/kernels/apache_toree_scala/bin/run.sh",
        "--profile",
        "{connection_file}"
      ],
      "env": {
        "DEFAULT_INTERPRETER": "Scala",
        "__TOREE_SPARK_OPTS__": "--jars=./lib/*.jar",
        "__TOREE_OPTS__": "",
        "SPARK_HOME": "/home/sergey/spark-3.3.2-bin-hadoop3"  
      },
      "display_name": "Apache Toree - Scala",
      "language": "scala",
      "interrupt_mode": "signal",
      "metadata": {}
    }