I'm devving a Storm project, and have been running it in local mode. I've been building code for the past couple days and just ran it this morning, and received this error in the console.
Working Directory = C:\dev\SE\SportsCloud\SEGOps\Storm
SLF4J: The requested version 1.5.8 by your slf4j binding is not compatible with [1.6, 1.7]
SLF4J: See http://www.slf4j.org/codes.html#version_mismatch for further details.
0 [main] INFO backtype.storm.zookeeper - Starting inprocess zookeeper at port 2000 and dir C:\Users\t-erros\AppData\Local\Temp\/400384ae-630f-41de-ba92-31b63969d91e
229 [main] INFO backtype.storm.daemon.nimbus - Starting Nimbus with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/6fc2a7e4-6545-4c1d-8506-0031fd0368ca", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" [6700 6701 6702 6703], "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
233 [main] INFO backtype.storm.daemon.nimbus - Using default scheduler
269 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
315 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
338 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
378 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
384 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
388 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
391 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
396 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
399 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
420 [main] INFO backtype.storm.daemon.supervisor - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/0ba572e6-1955-4708-8b48-600dcd95511d", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (1 2 3), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
435 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
442 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
444 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
472 [main] INFO backtype.storm.daemon.supervisor - Starting supervisor with id 415b310e-adb9-4f3e-addb-487b5dabd65d at host eross-pc.northamerica.corp.microsoft.com
477 [main] INFO backtype.storm.daemon.supervisor - Starting Supervisor with conf {"dev.zookeeper.path" "/tmp/dev-storm-zookeeper", "topology.tick.tuple.freq.secs" nil, "topology.fall.back.on.java.serialization" true, "topology.max.error.report.per.interval" 5, "zmq.linger.millis" 0, "topology.skip.missing.kryo.registrations" true, "ui.childopts" "-Xmx768m", "storm.zookeeper.session.timeout" 20000, "nimbus.reassign" true, "topology.trident.batch.emit.interval.millis" 50, "nimbus.monitor.freq.secs" 10, "java.library.path" "/usr/local/lib:/opt/local/lib:/usr/lib", "topology.executor.send.buffer.size" 1024, "storm.local.dir" "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/111ba4fe-03cb-4d65-8f92-6531a0f8ece9", "supervisor.worker.start.timeout.secs" 120, "topology.enable.message.timeouts" true, "nimbus.cleanup.inbox.freq.secs" 600, "nimbus.inbox.jar.expiration.secs" 3600, "drpc.worker.threads" 64, "topology.worker.shared.thread.pool.size" 4, "nimbus.host" "localhost", "storm.zookeeper.port" 2000, "transactional.zookeeper.port" nil, "topology.executor.receive.buffer.size" 1024, "transactional.zookeeper.servers" nil, "storm.zookeeper.root" "/storm", "supervisor.enable" true, "storm.zookeeper.servers" ["localhost"], "transactional.zookeeper.root" "/transactional", "topology.acker.executors" 1, "topology.transfer.buffer.size" 1024, "topology.worker.childopts" nil, "drpc.queue.size" 128, "worker.childopts" "-Xmx768m", "supervisor.heartbeat.frequency.secs" 5, "topology.error.throttle.interval.secs" 10, "zmq.hwm" 0, "drpc.port" 3772, "supervisor.monitor.frequency.secs" 3, "topology.receiver.buffer.size" 8, "task.heartbeat.frequency.secs" 3, "topology.tasks" nil, "topology.spout.wait.strategy" "backtype.storm.spout.SleepSpoutWaitStrategy", "topology.max.spout.pending" nil, "storm.zookeeper.retry.interval" 1000, "topology.sleep.spout.wait.strategy.time.ms" 1, "nimbus.topology.validator" "backtype.storm.nimbus.DefaultTopologyValidator", "supervisor.slots.ports" (4 5 6), "topology.debug" false, "nimbus.task.launch.secs" 120, "nimbus.supervisor.timeout.secs" 60, "topology.message.timeout.secs" 30, "task.refresh.poll.secs" 10, "topology.workers" 1, "supervisor.childopts" "-Xmx256m", "nimbus.thrift.port" 6627, "topology.stats.sample.rate" 0.05, "worker.heartbeat.frequency.secs" 1, "topology.acker.tasks" nil, "topology.disruptor.wait.strategy" "com.lmax.disruptor.BlockingWaitStrategy", "nimbus.task.timeout.secs" 30, "storm.zookeeper.connection.timeout" 15000, "topology.kryo.factory" "backtype.storm.serialization.DefaultKryoFactory", "drpc.invocations.port" 3773, "zmq.threads" 1, "storm.zookeeper.retry.times" 5, "topology.state.synchronization.timeout.secs" 60, "supervisor.worker.timeout.secs" 30, "nimbus.file.copy.expiration.secs" 600, "drpc.request.timeout.secs" 600, "storm.local.mode.zmq" false, "ui.port" 8080, "nimbus.childopts" "-Xmx1024m", "storm.cluster.mode" "local", "topology.optimize" true, "topology.max.task.parallelism" nil}
483 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
491 [main-EventThread] INFO backtype.storm.zookeeper - Zookeeper state update: :connected:none
495 [main] INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
509 [main] INFO backtype.storm.daemon.supervisor - Starting supervisor with id 2e0fd15c-80fc-4f31-a06e-481c55755164 at host eross-pc.northamerica.corp.microsoft.com
556 [main] INFO backtype.storm.daemon.nimbus - Received topology submission for xes-rtbi-stream-test with conf {"topology.max.task.parallelism" nil, "topology.acker.executors" 1, "topology.kryo.register" nil, "topology.kryo.decorators" (), "topology.name" "xes-rtbi-stream-test", "storm.id" "xes-rtbi-stream-test-1-1372358089", "topology.debug" false}
580 [main] INFO backtype.storm.daemon.nimbus - Activating xes-rtbi-stream-test: xes-rtbi-stream-test-1-1372358089
672 [main] INFO backtype.storm.scheduler.EvenScheduler - Available slots: (["2e0fd15c-80fc-4f31-a06e-481c55755164" 4] ["2e0fd15c-80fc-4f31-a06e-481c55755164" 5] ["2e0fd15c-80fc-4f31-a06e-481c55755164" 6] ["415b310e-adb9-4f3e-addb-487b5dabd65d" 1] ["415b310e-adb9-4f3e-addb-487b5dabd65d" 2] ["415b310e-adb9-4f3e-addb-487b5dabd65d" 3])
698 [main] INFO backtype.storm.daemon.nimbus - Setting new assignment for topology id xes-rtbi-stream-test-1-1372358089: #backtype.storm.daemon.common.Assignment{:master-code-dir "C:\\Users\\t-erros\\AppData\\Local\\Temp\\/6fc2a7e4-6545-4c1d-8506-0031fd0368ca/nimbus/stormdist/xes-rtbi-stream-test-1-1372358089", :node->host {"2e0fd15c-80fc-4f31-a06e-481c55755164" "eross-pc.northamerica.corp.microsoft.com"}, :executor->node+port {[1 1] ["2e0fd15c-80fc-4f31-a06e-481c55755164" 4]}, :executor->start-time-secs {[1 1] 1372358089}}
1513 [Thread-8] INFO backtype.storm.daemon.supervisor - Downloading code for storm id xes-rtbi-stream-test-1-1372358089 from C:\Users\t-erros\AppData\Local\Temp\/6fc2a7e4-6545-4c1d-8506-0031fd0368ca/nimbus/stormdist/xes-rtbi-stream-test-1-1372358089
1861 [Thread-8] INFO backtype.storm.daemon.supervisor - Extracting resources from jar at C:\Users\t-erros\Documents\Jars\commons-configuration-1.9-bin\commons-configuration-1.9\commons-configuration-1.9-javadoc.jar to C:\Users\t-erros\AppData\Local\Temp\/111ba4fe-03cb-4d65-8f92-6531a0f8ece9/supervisor/stormdist/xes-rtbi-stream-test-1-1372358089/resources
1870 [Thread-8] ERROR backtype.storm.event - Error when processing event
java.io.IOException: Cannot run program "unzip" (in directory "."): CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at org.apache.commons.exec.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
at org.apache.commons.exec.DefaultExecutor.launch(DefaultExecutor.java:254)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:319)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:160)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:147)
at backtype.storm.util$exec_command_BANG_.invoke(util.clj:324)
at backtype.storm.util$extract_dir_from_jar.invoke(util.clj:329)
at backtype.storm.daemon.supervisor$fn__4803.invoke(supervisor.clj:452)
at clojure.lang.MultiFn.invoke(MultiFn.java:172)
at backtype.storm.daemon.supervisor$mk_synchronize_supervisor$this__4716.invoke(supervisor.clj:295)
at backtype.storm.event$event_manager$fn__2507.invoke(event.clj:24)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(Unknown Source)
at java.lang.ProcessImpl.start(Unknown Source)
... 15 more
1878 [Thread-8] INFO backtype.storm.util - Halting process: ("Error when processing an event")
I commented out everything in my main function and I'm left with this:
public class XESRealTimeOITopology
{
public static void main(String[] args) throws Exception
{
System.out.println("Working Directory = " + System.getProperty("user.dir"));
TopologyBuilder builder = new TopologyBuilder();
Config conf = new Config();
conf.setDebug(false);
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("xes-rtbi-stream-test", conf, builder.createTopology());
boolean loop = true;
while (loop)
{
Thread.sleep(60000);
}
cluster.shutdown();
}
}
The exception is thrown at the cluster.submitToplogy line. My coworker has the exact same project setup on his computer, also commented the lines out and it works fine. I have downloaded the unzip program but I'm not sure where to put it, or why this error is even being thrown. Does anyone have any ideas?
Apparently storm isn't fond of external jars. I had them in the Reference Library, moved them all into the user-defined Storm Lib and everything is a-okay. Thanks for the answers either way! :)