call python script function from scala

Question

Attempt to call python function from scala fails with below error. But works fine when the same command is invoked directly from command line.

Please find below simplified code snippets :-

greeting.py

import logging
import os


def greet(arg):
    print("hey " + arg)

StraightPyCall.scala

package git_log

object StraightPyCall {

  def main(args: Array[String]): Unit = {

    val commandWithNewLineInBeginning =
      """
        |python -c "import sys;sys.path.append('~/playground/octagon/bucket/pythonCheck'); from greeting import *; greet('John')"
        |""".stripMargin

    //new line stripped out from beginning and end
    val executableCommand = commandWithNewLineInBeginning.substring(1, commandWithNewLineInBeginning.length - 1)

    println("command is :-")
    println(executableCommand)

    import sys.process._

    s"$executableCommand".!!
  }
}

output of above scala program is :-

command is :-
python -c "import sys;sys.path.append('~/playground/octagon/bucket/pythonCheck'); from greeting import *; greet('John')"

  File "<string>", line 1
    "import
          ^
SyntaxError: EOL while scanning string literal
Exception in thread "main" java.lang.RuntimeException: Nonzero exit value: 1
    at scala.sys.package$.error(package.scala:26)
    at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.slurp(ProcessBuilderImpl.scala:134)
    at scala.sys.process.ProcessBuilderImpl$AbstractBuilder.$bang$bang(ProcessBuilderImpl.scala:104)
    at git_log.StraightPyCall$.main(StraightPyCall.scala:19)
    at git_log.StraightPyCall.main(StraightPyCall.scala)

When I tried executing the command that is printed on the console. It works perfectly fine.

python -c "import sys;sys.path.append('~/playground/octagon/bucket/pythonCheck'); from greeting import *; greet('John')"

Result:-

hey John

Note : Below is the ProcessBuilder toString representation (copied from stacktrace while debugging) :-

[python, -c, "import, sys;sys.path.append('/Users/mogli/jgit/code-conf/otherScripts/pythonScripts/CallRelativePyFromBash/pyscripts');, from, Greet, import, *;, greet_with_arg('John')"]

Kindly suggest, what needs to be modified in commandWithNewLineInBeginning to make it work from scala

Answer 1

It works from the command line because the shell is parsing and interpreting the string before invoking the python command. In the Scala code the ProcessBuilder is trying to parse and interpret the string without the shell's help.

We can help the interpreting. This should work.

Seq("python"
   ,"-c"
   ,"import sys;sys.path.append('~/playground/octagon/bucket/pythonCheck'); from greeting import *; greet('John')"
   ).!!

If you really have to start out with the full string then maybe you can break it up before processing.

For example: If you know that the pattern is always "cmnd -c string" then this might work.

commandWithNewLineInBeginning.replaceAll("\"","")
                             .split("((?=-c)|(?<=-c))")
                             .map(_.trim)
                             .toSeq.!!