Search code examples
apache-kafkaapache-zookeeperapache-stormtopologybigdata

Config cluster ZK_HOST on Storm Topology


Now I have just set up successfully my storm topology with single node on single machine. I use KafkaSpout as below:

String zkHostPort = "localhost:2181";
String topic = "sentences";

String zkRoot = "/kafka-sentence-spout";
String zkSpoutId = "sentence-spout";
ZkHosts zkHosts = new ZkHosts(zkHostPort);

SpoutConfig spoutCfg = new SpoutConfig(zkHosts, topic, zkRoot, zkSpoutId);
KafkaSpout kafkaSpout = new KafkaSpout(spoutCfg);
return kafkaSpout;

Now I set up cluster zookeeper(three node: server1.com:2181, server2.com:2181. server3.com:2181) and cluster kafka (three node). I wonder how I can change code on Storm Topology for this purpose. Please help me!!


Solution

  • Please, use the configuration below:

    String zkHostPort = "server1.com:2181,server2.com:2181,server3.com:2181";
    String topic = "sentences";
    
    String zkRoot = "/kafka-sentence-spout";
    String zkSpoutId = "sentence-spout";
    ZkHosts zkHosts = new ZkHosts(zkHostPort);
    
    SpoutConfig spoutCfg = new SpoutConfig(zkHosts, topic, zkRoot, zkSpoutId);
    KafkaSpout kafkaSpout = new KafkaSpout(spoutCfg);
    return kafkaSpout;
    

    Note: the most common issue here is space placed between hosts after comma, there must not be space between hosts.

    Correct:

    server1.com:2181,server2.com:2181,server3.com:2181

    Wrong:

    server1.com:2181, server2.com:2181, server3.com:2181