Search code examples
scalaapache-sparkspark-submitspark-jdbc

How to fix initialization of Logger error while using spark-submit command

I've got the problem when run my spark-jdbc job to connect to another db. But I've got error before. Exception in thread "main" java.lang.AbstractMethodError at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:99)

My Logger wesn't able to be initialized by scala. I'm using scala 2.11 and spark with the same versions.

Can't debug this issue via IDE, cause there all is fine, but when I run spark-submit, then error happens.

Solution

  • Got the same error while using Spark 2.3 and I should've been using Spark 2.2. Apparently that method was made abstract in the later version so I was getting that error.

    Try this