Search code examples
scalaapache-sparkself

what is the "self" meaning in new class of scala


recently I am reading the source of spark. When reaching to the class of "org.apache.spark.deploy.SparkSubmit", I got confusion about the keyword "self" and the operator "=>". Is anyone can explain me for that?

override def main(args: Array[String]): Unit = {
val submit = new SparkSubmit() {
  self =>

  override protected def parseArguments(args: Array[String]): SparkSubmitArguments = {
    new SparkSubmitArguments(args) {
      override protected def logInfo(msg: => String): Unit = self.logInfo(msg)

      override protected def logWarning(msg: => String): Unit = self.logWarning(msg)
    }
  }

  override protected def logInfo(msg: => String): Unit = printMessage(msg)

  override protected def logWarning(msg: => String): Unit = printMessage(s"Warning: $msg")

  override def doSubmit(args: Array[String]): Unit = {
    try {
      super.doSubmit(args)
    } catch {
      case e: SparkUserAppException =>
        exitFn(e.exitCode)
      case e: SparkException =>
        printErrorAndExit(e.getMessage())
    }
  }

}

BTW: this question is totally different from the “duplicated one”. Although these two are very same, what i am asking is about the “self =>” near the key word of “new class” rather than the “duplicated” with “ some name =>” in the class definition of scala. it’s not a same question


Solution

  • The statement

    self =>
    

    is called a "self type annotation" and it creates a value named self that refers to the instance of the class being constructed. This can be used in places where the this value for the class is not available. In particular, it can be used inside a nested class, where this refers to the nested class and a reference to the outer class is not automatically available.

    In your case, self is used here:

    new SparkSubmitArguments(args) {
      override protected def logInfo(msg: => String): Unit = self.logInfo(msg)
    
      override protected def logWarning(msg: => String): Unit = self.logWarning(msg)
    }
    

    This makes the new instance of SparkSubmitArguments use the logInfo and logWaringing methods from the outer, containing class. You can't use this at this point of the code because it would refer to the inner class, not the outer class. (If you do use this here you will get an infinite loop)