Search code examples
apache-storm

Is there a way to pass a variable from a Bolt to a Spout in Apache Storm?


In storm the flow of information (tuples) is from Spout to Bolt. To prevent information overload, I am filtering most of the data at the spout in the beginning, but after processing of data, I want to add some more information pass through the spout based on the pattern in the data. In other words, I want to dynamically change the information passed from a Spout at run-time based on the data processed by the bolts so far.


Solution

  • No. But you can move the filtering logic out of the spout into a new first bolt. The spout just fetches all data and forwards it to the new filter bolt. For bolts it is possible to have cycles in the graph, ie, you can feed back information to the filter bolt. For example something like this:

    builder.addSpout("spout",...);
    builder.addBolt("filter",...)
      .localOrShuffleGrouping("spout") // regular forward connection
      .allGrouping("someLaterBolt"); // cyclic feedback connection 
    // add more bolts here
    builder.addBolt("someLaterBolt",...).someConnectionPattern("somePreviousBolt")