recently I am reading the source of spark. When reaching to the class of "org.apache.spark.deploy.SparkSubmit", I got confusion about the keyword "self" and the operator "=>". Is anyone can explain me for that?
override def main(args: Array[String]): Unit = {
val submit = new SparkSubmit() {
self =>
override protected def parseArguments(args: Array[String]): SparkSubmitArguments = {
new SparkSubmitArguments(args) {
override protected def logInfo(msg: => String): Unit = self.logInfo(msg)
override protected def logWarning(msg: => String): Unit = self.logWarning(msg)
}
}
override protected def logInfo(msg: => String): Unit = printMessage(msg)
override protected def logWarning(msg: => String): Unit = printMessage(s"Warning: $msg")
override def doSubmit(args: Array[String]): Unit = {
try {
super.doSubmit(args)
} catch {
case e: SparkUserAppException =>
exitFn(e.exitCode)
case e: SparkException =>
printErrorAndExit(e.getMessage())
}
}
}
BTW: this question is totally different from the “duplicated one”. Although these two are very same, what i am asking is about the “self =>” near the key word of “new class” rather than the “duplicated” with “ some name =>” in the class definition of scala. it’s not a same question
The statement
self =>
is called a "self type annotation" and it creates a value named self
that refers to the instance of the class being constructed. This can be used in places where the this
value for the class is not available. In particular, it can be used inside a nested class, where this
refers to the nested class and a reference to the outer class is not automatically available.
In your case, self
is used here:
new SparkSubmitArguments(args) {
override protected def logInfo(msg: => String): Unit = self.logInfo(msg)
override protected def logWarning(msg: => String): Unit = self.logWarning(msg)
}
This makes the new instance of SparkSubmitArguments
use the logInfo
and logWaringing
methods from the outer, containing class. You can't use this
at this point of the code because it would refer to the inner class, not the outer class. (If you do use this
here you will get an infinite loop)