Search code examples
scalaapache-sparkbigdata

Match case in Scala with tail


I'm new to Scala and finding the below code hard to understand. It would be really helpful if you guys can explain me in simple words what parseArguments is doing for "--config-path" and what is ::path::tail doing. we are getting scala.MatchError: None (of class scala.None$) error when we are trying to match variable configPathArg.

Here we are passing value to --config-path when we are submitting the spark-submit command but we got none matching error. Could you please let me know why it is giving error even though we are passing value to --config-path?

Thanks in advance :)

def parseArguments(map: Map[String, String], list: List[String]): Map[String, String] = { list match {
        case Nil => map
        case ("--config-path" | "-cp") :: path :: tail =>
          parseArguments(map ++ Map(configPathArg -> path), tail)
        case unknown :: tail =>
          parseArguments(map, tail)
        }
      }

val arguments = parseArguments(Map(), args.toList)
arguments.get(configPathArg) match {
case some(configPath: String) =>
 val is = hfs.open(new Path(configPath))
 }

Solution

  • The syntax head :: tail matches a List where the 1st element is head and the rest is tail (the rest can be empty, i.e. Nil).

    In your case "--config-path" :: path :: tail means you're matching a List where there's at least 2 elements and the 1st one is the string --config-path and the second one is any value but will then be available as path.

    • List("--config-path", "whatever") would match
      • and path would have whatever as value
    • List("--config-path", "whatever", ...) would match
      • and path would have whatever as value
    • List("another string", "whatever") would NOT match
    • a list of 0 or 1 item would NOT match