Search code examples
apache-storm

How to monitor the tuple size in byte in apache storm?


I want to know about the size of data, which is transferred among all the bolts and spouts. Does anyone have any idea how it is possible to find the tuple size in Byte?


Solution

  • You can enable the topology.serialized.message.size.metrics. The documentation says:

    Enable tracking of network message byte counts per source-destination task. This is off by default as it creates tasks2 metric values, but is useful for debugging as it exposes data skew when tuple sizes are uneven.

    Then your sizes should be available in standard metrics.

    Do note that bolts collocated on a single worker obviously transfer their tuples locally, without any serialization, and therefore their transfer sizes will be 0.