Search code examples
apache-flink

How to get the latency number of flink job

I know flink collects latency metrics out of box, but I want to know how can I get the latency data of my job.

Do I have to use custom reporter(e.g. graphite) to view the latency?

I saw there is latency metrics in flink dashboard, but no data when I added the latency of a sink operator. If this is not the latency data, what is it?

Thank you.

I am using Flink V1.2.1 by the way.

Solution

  • The Flink dashboard can only display single values. But you can examine the latency by using the REST api. The URL should look like this:

    http://localhost:8081/jobs/f184a26ee033d5448042c18de57499a1/vertices/91f34222e409e97879611b1fd334816e/metrics?get=0.Sink__Unnamed.latency

    And the reply will look like this:

    [
  {
    id: "0.Sink__Unnamed.latency",
    value: "{LatencySourceDescriptor{vertexID=1, subtaskIndex=0}={p99=416.0699999999999, p50=253.0, min=102.0, max=421.0, p95=378.04999999999995, mean=249.953125}}"
  }
]

    You can use http://localhost:8081/jobs/ to find your job id, and then http://localhost:8081/jobs/<job-id>/vertices/ to choose an operator in your execution graph. http://localhost:8081/jobs/<job-id>/vertices/<operator-id>/metrics will show the available metrics.