Search code examples
pythonapache-storm

What the sync method is doing and what is the frequency of heartbeat tuple?

I am using Storm 0.9.3, using Python.

I'm referring to Multilang-protocol.

Can anyone explain:

  • what the sync() is doing method?
  • what is the frequency of heartbeat tuple?
Solution

  • Storm 0.9.3 introduced multilang subprocess heartbeat. (Actually I introduced it.)

    As you already know, nimbus and supervisor check worker's heartbeat and take proper action when worker didn't write its heartbeat in time.

    Multilang subprocess heartbeat is doing similar thing, but it uses multilang protocol instead of ZK.

    1. Before 0.9.3, sync() is only used for ShellSpout to notify subprocess's execution of next() has been finished. After 0.9.3, Storm added new meaning to sync(), "heartbeat" (means that notifying I'm OK).

    2. ShellSpout doesn't send heartbeat tuple, cause normal circumstance Spout's nextTuple() is called frequently, so does subprocess's next(). ShellBolt sends heartbeat tuple to subprocess every 1 sec.

    Hope this helps.