I have created a storm cluster over Amazon EC2 instances. I tested my code in local Environment (pseudo-distributed environment) and it worked fine. But the code doesn't seems to work when I run the code in distributed environment. Is there something technical I am missing here, while setting up the cluster.
I can make a statement that the supervisor nodes are not running the bolts, as I checked the CPU usage of all the supervisor nodes and none of them was going above 1% but the master(nimbus) was running at 100% every time until the code execution finished.
I also checked the Java processes running in supervisor nodes and "jps" doesn't show my topology.
Note: The cluster shows 2 supervisors in the UI running over nimbus. That shows that the cluster is setup in distributed environment.
Solution to my Problem:
Previously I was running Using the LocalCluster
instead of StormSubmitter
Use this:
StormSubmitter.submitTopologyWithProgressBar(args[0], conf, builder.createTopology());
instead of:
LocalCluster cluster = new LocalCluster();
cluster.submitTopology("test", conf, builder.createTopology());
Utils.sleep(10000);
cluster.killTopology("test");
cluster.shutdown();