Search code examples
mqttinfluxdbtelegraf

Telegraf not writing data to any output type

I am trying to consume MQTT events and publishing them via Telegraf. Eventually I'll only use InfluxDB as the output but for debugging reasons I'm also writing to a file now.

The problem is that the file generated by Telegraf is always empty, and nothing is written to InfluxDB even though MQTT has events being generated.

An event published in MQTT looks like this:

{"100001":"100","100002":"200"}

The telegraf.conf file looks like this:

    [[inputs.mqtt_consumer]]
      servers = ["tcp://mqtt:1883"]
      qos = 0
      connection_timeout = "30s"
      topics = [
        "mytopics/test",
      ]
      persistent_session = true
      client_id = "MQTT"
      data_format = "json"
      data_type = "integer"
 
    
    [[outputs.file]]
      files = ["stdout", "./test.txt"]
      data_format = "json"

    [[outputs.influxdb]]
    database = "testdb"
    urls = ["http://influxdb:8086"]

    [agent]
    interval = "10s"
    debug = true
    quiet = false

For example, If I change the telegraf.conf file for debugging purposes and remove data_format = "json", the following error happens:

Error in plugin [inputs.mqtt_consumer]: metric parse error: expected tag at offset 31: "{\"100001\":\"100\",\"100002\":\"200\"}"

After fixing it and adding data_format = "json", nothing happens and the messages in Debug Mode only show:

 Output [file] buffer fullness: 0 / 10000 metrics.

I am wondering what is wrong with my configuration, as I have tried writing this data to InfluxDB, files and Prometheus using Telegraf without any luck. Also, there are no error messages and everything looks OK.

Any ideas are welcome!

Solution

  • The issue is that the mqtt topic is getting read, but telegraf is unable to parse the data that is coming in without some more knowledge of the data.

    With the data you are receiving from MQTT are you looking to store JSON elements as fields like so: mqtt 100001=100i,100002=200i? For example, I can produce a metric like that with the json_v2 parser:

    [[inputs.file]]
  files = ["data.json"]
  data_format = "json_v2"
  [[inputs.file.json_v2]]
    [[inputs.file.json_v2.field]]
      path = "100001"
      type = "int"

    [[inputs.file.json_v2.field]]
      path = "100002"
      type = "int"

    Obviously, you will want to replace the file with mqtt for your config. With the example data you provided, this produces:

    file 100001=100i,100002=200i 1659117305000000000