We are using Flink 1.9.1 on EMR 5.29 with multi-master mode enabled. We added the below configuration to the flink-conf.yaml file in conf folder and added the jar flink-metrics-statsd-1.9.1.jar to the flink/lib folder.
metrics.scope.jm: <host>.jobmanager
metrics.scope.jm.job: <host>.jobmanager.<job_name>
metrics.scope.tm: <host>.taskmanager.<tm_id>
metrics.scope.tm.job: <host>.taskmanager.<tm_id>.<job_name>
metrics.scope.task: <host>.taskmanager.<tm_id>.<job_name>.<task_name>.<subtask_index>
metrics.scope.operator: <host>.taskmanager.<tm_id>.<job_name>.<operator_name>.<subtask_index>
metrics.reporters: stsd
metrics.reporter.stsd.factory.class: org.apache.flink.metrics.statsd.StatsDReporter
metrics.reporter.stsd.host: localhost
metrics.reporter.stsd.port: 8130
metrics.reporter.stsd.interval: 60 SECONDS
When we listen to the statsD port, we don't get any of the metrics. Is there anything that we are missing out? Is there a way to check if some parameter is not getting passed?
As suggested in one of the other answers, we tried giving the IP address of the master also. Does not seem to work.
Found the issue :
This should have been used:
metrics.reporter.stsd.class: org.apache.flink.metrics.statsd.StatsDReporter
Instead of:
metrics.reporter.stsd.factory.class: org.apache.flink.metrics.statsd.StatsDReporter