Search code examples
apache-kafkadatadog

kafka datadog not sending metrics correctly


i am trying to send kafka consumer metrics to datadog but its not showing in monitoring when I select the node. The server is giving below check in status

   Instance ID: kafka_consumer:d6........f5 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/kafka_consumer.d/conf.yaml
      Total Runs: 567
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 162ms
      Last Execution Date : 2021-01-14 10:49:06.000000 UTC
      Last Successful Execution Date : 2021-01-14 10:49:06.000000 UTC
      metadata:
        version.major: 2
        version.minor: 5
        version.patch: 0
        version.raw: 2.5.0
        version.scheme: semver


JMXFetch
    runtime_version : 11.0.9.1
    version : 0.40.3
  Initialized checks
    kafka
      instance_name : kafka-10.128.0.105-9999
      message : <no value>
      metric_count : 99
      service_check_count : 0
      status : OK
Failed checks
    no checks

JMX is as above. Please help in finding what could be wrong.


Solution

  • Nothing is wrong there. DataDog conceals that the Kafka integration uses Dogstatsd under the hood. When use_dogstatsd: 'true within /etc/datadog-agent/datadog.yaml is set, metrics do appear in DataDog webUI. If that option is not set the default Broker data is available via JMXFetch using sudo -u dd-agent datadog-agent status as also via sudo -u dd-agent datadog-agent check kafka but not in the webUI.