Search code examples
intellij-ideaapache-sparksbtclassloadergetresource

log4j properties file bundled into jar in spark app is ignored

I need to read a custom log4j.properties from src/resources and this is not working

try{
  val  inStream :InputStream= className.this.getClass.getClassLoader.getResourceAsStream("log4j.properties");

  logCfgProps.load(inStream)

} catch {
  case e: Throwable=>
    e.printStackTrace()
    log.error("log.properties file not present")
}

PropertyConfigurator.configure(logCfgProps)

meaning that the log4j bundled in the jar is ignored.

I cannot touch the log4j properties in the conf directory in the spark home.

What are the other options?

EDIT There must be something wrong (in the classpath ?) since doing this

val resource:URL = Thread.currentThread().getContextClassLoader()
  .getResource("log4j.properties");
System.out.println("resource = " + resource);

It points to the log4j in the conf directory that I cannot modify and that I need to ignore.

How to do that?

Solution

    1. You can just give another name to your file (and using it in the code as well), so there is no conflict with conf/log4j.properties.

    2. You can get all resources with the given name on the classpath using ClassLoader.getResources("log4j.properties") and choose which one you want.