Search code examples
apache-flinkflink-streaming

Add custom data to the Flink metrics

In Apache Flink exists 4 types of metrics. The goal is to add additional info to the histogram metric: user_id and speed_of_processing For example:

user_id: speed_of_processing

1:30 sec
2:15 sec

... How can I add such type of information?

Solution

  • It looks like you want to measure end-to-end latency and report it separately for each user_id. That's not a good match for flink's metrics -- it's not a good idea to have per-key (e.g., per user) metrics -- Flink's metrics system isn't designed with that sort of scale in mind.

    You could instead perform these measurements and send the results to a sink. Or use a histogram metric that only reports results aggregated across all users.