Search code examples
scalapureconfig

Scala: Creating generic util with PureConfig


I am looking for some utilities in scala which fit akka well that can map HOCON to Scala class/object. (Something like @Configuration in Spring mapping the .yml or .properties file to Configuration bean/Java Class.)

What I tried with PureConfig :

spark.conf config file in resources

spark{
    master {
        host = 1.2.3.4
        port = 7077
    }
}

Mapping to following scala classes :

trait Configuration {
    val nameSpace: String
}

case class SparkConfig(master: SparkMasterConfig) extends Configuration {
    override val nameSpace: String = "spark"
}

case class SparkMasterConfig(host: String,
                                    port: Int)

PureConfig works fine without generic:

import pureconfig.generic.auto._
import com.typesafe.config.ConfigFactory

val conf = ConfigFactory.parseResources("spark.conf")
val sparkConfig = pureconfig.loadConfig[SparkConfig]("spark")

val config = sparkConfig match {
    case Left(f) => fail(f.toString)
    case Right(c) => c
}

However the following generic util doesn't even compile with not enough arguments for method error

object PureConfigLoader{
    def load[T <: Configuration](clazz: Class[T]): T = {
        val nameSpace = clazz.getField("nameSpace").get().asInstanceOf[String]

        import pureconfig.generic.auto._
        val configResult = pureconfig.loadConfig[T](nameSpace)  // this doesn't compile

        configResult match {
            case Right(x) => x.asInstanceOf[T]
            case Left(x) => throw new IllegalArgumentException(s"Fail to parse ${clazz.getSimpleName} from namespace $nameSpace")
        }
    }
}

val config = PureConfigLoader.load(classOf[SparkConfig])

My questions are:

  1. What can I do with this generic PureConfig util?
  2. import pureconfig.generic.auto._ always marked as unused import by IntelliJ and it will be removed when formatting my code, how to fix this?
  3. Are there any other config util/library works fine in similar way? I have also tried circe-config but still got similar issues. Simple Scala Config uses Scala's Dynamic which is not good for refactoring like renaming the configuration property field.

Thank you


Solution

  • I use a small wrapper of Typesafe config:

    https://github.com/kxbmap/configs

    I tried your example - this compiles:

      object PureConfigLoader{
        def load[T <: MyConfig[T]](clazz: Class[T])(implicit A: Configs[T]): T = {
          val nameSpace = clazz.getField("nameSpace").get().asInstanceOf[String]
    
          val config = ConfigFactory.load("spark.conf")
            Configs[T].get(config, nameSpace) 
              .valueOrThrow(e=>
                 new IllegalArgumentException(s"Fail to parse ${clazz.getSimpleName} from namespace $nameSpace: ${e.messages}" 
              )
          )
        }
      }
    
      val sparkConfig = PureConfigLoader.load(classOf[SparkConfig])
    
      abstract class MyConfig[T] (implicit A: Configs[T])
    
      case class SparkConfig() extends MyConfig[SparkConfig]
    

    Update

    I just saw in our code that you might not need this Util class at all! We use:

    val config = ConfigFactory.load("spark.conf")
    val sparkConfig = Configs[SparkConfig].get(config, nameSpace)