How to convert a shell script (bash) to Scala code?

Question

I have a shell script with series of defined steps. I am trying to convert the script to Scala code by maintaining the order of steps. Basically I want my Scala code to be a mirror of the shell script.

I've used sys.process._ library. The shell script has commands like:

mkdir <pathToDir>
hadoop fs -copyToLocal <source> <dest>
rm -r <pathToDir>
java -jar <someScript>

I need to maintain the order of execution of these steps.

I tried something like:

import sys.process._
Class A {
  def processMethod(): Unit = { 
    Process("mkdir -p <dir1>") #| Process("hadoop fs -copyToLocal 
    <hdfsDir1> <localDir1>") #| Process("mkdir -p <dir2>") #| 
    Process("hadoop fs -copyToLocal <hdfsdir2>/*.parquet <localDir2>") 
    #| Process("java -jar <pathToJarFile> -script <sampleScripts> 1>&2") 
    #| Process("rm -r<localDir1>") #| Process("rm -r <localDir2>") ! 
  }
}

I'm expecting the operations to execute in the order they have been defined. I'm confused about how ProcessBuilder/Process works or if there is an alternative to convert this whole thing to Scala code?

Answer 1

According to the document, #| constructs a command that will run command and pipes the output to other. Thats being said, the following code in Scala:

(Process("echo hello") #| Process("wc -c")).!

is equivalent to following Bash code:

echo hello | wc -c

which is not what you want.

What you are looking for is ### operator which constructs a command that will run one command and then other.

Using this operator, you can do write following Scala code:

(Process("ls") ### Process("echo hello")).!

Which is equivalent to following Bash code:

ls
echo hello

However, note that using Process in the way above is not completely bash equivalent because it cannot change current directory with cd nor use bash syntax such as if, for, case.

If you really want bash equivalent code, the only way is running the script with bash.