Search code examples
javaapache-kafkaapache-stormbolt

Storm bolt executes more than its parent

I have a Topology which contains a KafkaSpout and 2 bolts.

BoltParseJsonInput and its execute method:

public void execute(Tuple input) {
    // TODO Auto-generated method stub
    String data = input.getString(4);
    js = new JSONObject(data);

    String userId = js.getString("userId");
    String timestamp = js.getString("timestamp");
    counter++;
    System.out.println(counter);
    collector.emit(input, new Values(userId, timestamp));
    collector.ack(input);
}

BoltInsertRedis and its execute method

    public void execute(Tuple input) {
    // TODO Auto-generated method stub
    String userId = input.getStringByField("userId");
    int timestamp = 0;
    try {

        timestamp = convertTimestampToEpoch(input.getStringByField("timestamp"));
    } catch (ParseException e1) {
        // TODO Auto-generated catch block
        e1.printStackTrace();
    }

    String timestep = this.prefix + timestamp/10;
    String curTimestamp = jedis.hget(timestep, userId);
    if(curTimestamp == null || Integer.parseInt(curTimestamp) < timestamp) {
        jedis.hset(timestep, userId, Integer.toString(timestamp));
    }
    collector.ack(input);
}

BoltInsertRedis get the input from BoltParseJsonInput

builder.setBolt("ParseJsonInput-Bolt", new BoltParseJsonInput()).shuffleGrouping("Kafka-Spout");
    builder.setBolt("BoltRedisUserLastActive-Bolt", new BoltRedisUserLastActive()).shuffleGrouping("ParseJsonInput-Bolt");

But when I submit this topology into Storm, BoltInsertRedis execute more than BoltParseJsonInput storm ui

Can you explain to me what is the problem here?

Solution

  • I found that my ParseJsonBolt had made an exception at message 25700 and it keeps replaying execution at that point. When I made a try catch, it works well