Search code examples
build.gradleapache-storm

Topology not working due to Found multiple defaults.yaml resources when deploying to remote cluster


I have a single node remote cluster set up (all nimbus , supervisor , zookeeper) running on the same machine . I deployed my Topology (simple Exclamation Topology) to this remote cluster . While topology and jar got submitted successfully , nothing is happening in the cluster .

When I checked the supervisor logs , I could see this : 2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO] 42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started

Worker log files are empty .

On debugging a little in supervisor logs , I could see Launching worker with command: (Some java command) ..Firing this java command I can see this error :

Caused by: java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar.

I debugged more on internet and other stuff , and modified my build.gradle file too but still the same error whenever I deploy my topology .

This is my gradle file

dependencies {

    compile group: 'org.springframework.boot', name: 'spring-boot-starter-actuator', version: springBootVersion

    compile group: 'org.quartz-scheduler', name: 'quartz', version: quartzVersion

compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version: cljStackTrace

compile group: 'org.apache.storm' , name: 'storm-core',version: stormVersion

    ext {

      fatJarExclude = true

  }

}
task uberjar(type: Jar) {

from files(sourceSets.main.output.classesDir)

from {configurations.compile.collect {zipTree(it)}} {

    exclude "META-INF/*.SF"

    exclude "META-INF/*.DSA"

    exclude "META-INF/*.RSA"

    exclude "META-INF/LICENSE"
}
manifest {

attributes 'Main-Class': 'storm.topology.ExclamationTopology'
}
}

Solution

  • The jar file must not contain defaults.yaml file. Thus, you need to exclude it via

    exclude "defaults.yaml"
    

    Actually I would recommend the exclude all Storm dependencies from your jar. They are not needed and increase the size of the fat jar unnecessarily.