How to set spark standalone master url in play-framework conf/application.conf file?

Question

plugging the play application with spark standalone cluster it executes well on dev mode but when trying to deploy in production mode it gives following error:

 Caused by: org.apache.spark.SparkException: A master URL must be set in your configuration

I am using spark-2.1 here is the snippet

lazy val spark = SparkSession.builder().appName("Spark_with_Play").master("spark://ip:7077").config("spark.executor.memory", "2g").config("spark.deploy.defaultCores",8).getOrCreate()

So how can i set spark master in conf/appliaction.conf file in production mode ?

Answer 1

After many trials able to find typesafe config which is used to read the configurations from application.conf file and pass those into code. here is the content of application.conf file:

spark.master = "spark://ip:7077"

here is the line which is added to build.sbt

"com.typesafe"%"config"%"1.2.0"

in controller here is code the snippet:

import com.typesafe.config._
val conf = ConfigFactory.load()
val sparkMaster = conf.getString("spark.master")

val spark= SparkSession.builder().master(sparkMaster).getOrCreate()