I'm new to Scala and finding the below code hard to understand. It would be really helpful if you guys can explain me in simple words what parseArguments is doing for "--config-path" and what is ::path::tail doing. we are getting scala.MatchError: None (of class scala.None$) error when we are trying to match variable configPathArg.
Here we are passing value to --config-path when we are submitting the spark-submit command but we got none matching error. Could you please let me know why it is giving error even though we are passing value to --config-path?
Thanks in advance :)
def parseArguments(map: Map[String, String], list: List[String]): Map[String, String] = { list match {
case Nil => map
case ("--config-path" | "-cp") :: path :: tail =>
parseArguments(map ++ Map(configPathArg -> path), tail)
case unknown :: tail =>
parseArguments(map, tail)
}
}
val arguments = parseArguments(Map(), args.toList)
arguments.get(configPathArg) match {
case some(configPath: String) =>
val is = hfs.open(new Path(configPath))
}
The syntax head :: tail
matches a List
where the 1st element is head
and the rest is tail
(the rest can be empty, i.e. Nil
).
In your case "--config-path" :: path :: tail
means you're matching a List
where there's at least 2 elements and the 1st one is the string --config-path
and the second one is any value but will then be available as path
.
List("--config-path", "whatever")
would match
path
would have whatever
as valueList("--config-path", "whatever", ...)
would match
path
would have whatever
as valueList("another string", "whatever")
would NOT match