I have a dockerised telegraf (influxdata-docker/telegraf/1.21 at master · influxdata/influxdata-docker · GitHub) and influxdb (influxdata-docker/Dockerfile at master · influxdata/influxdata-docker · GitHub).
I’m trying to send data to influxdb from ttn but without any success. The config file for telegraf is:
[agent]
interval = "30s"
debug = true
quiet = false
logfile = "/var/log/telegraf/telegraf.log"
[[inputs.mqtt_consumer]]
servers = ["tcp://eu1.cloud.thethings.network:1883"]
qos = 0
connection_timeout = "30s"
topics = [ "+/devices/+/up" ]
client_id = "_my_id"
username = "my_username"
password = "my_password"
data_format = "json"
[[outputs.influxdb_v2]]
urls = ["http://172.21.0.2:8086"]
token = "my_token"
organization = "iot"
bucket = "ttn_data"
[[outputs.file]]
files = ["stdout", "/var/log/telegraf/metrics.out"]
use_batch_format = false
rotation_interval = "0h"
rotation_max_size = "0MB"
rotation_max_archives = 5
data_format = "influx"
It seems that the dockers have been setup properly and one sees the other:
Starting datahandler_influxdb_1 ... done
Starting datahandler_telegraf_1 ... done
Attaching to datahandler_influxdb_1, datahandler_telegraf_1
influxdb_1 | 2022-03-25T21:16:33.734688793Z info found existing boltdb file, skipping setup wrapper {"system": "docker", "bolt_path": "/var/lib/influxdb2/influxd.bolt"}
influxdb_1 | ts=2022-03-25T21:16:33.895365Z lvl=info msg="Welcome to InfluxDB" log_id=0_SjrEuW000 version=2.1.1 commit=657e1839de build_date=2021-11-09T03:03:48Z
influxdb_1 | ts=2022-03-25T21:16:33.899857Z lvl=info msg="Resources opened" log_id=0_SjrEuW000 service=bolt path=/var/lib/influxdb2/influxd.bolt
influxdb_1 | ts=2022-03-25T21:16:33.900228Z lvl=info msg="Resources opened" log_id=0_SjrEuW000 service=sqlite path=/var/lib/influxdb2/influxd.sqlite
influxdb_1 | ts=2022-03-25T21:16:33.912460Z lvl=info msg="Checking InfluxDB metadata for prior version." log_id=0_SjrEuW000 bolt_path=/var/lib/influxdb2/influxd.bolt
influxdb_1 | ts=2022-03-25T21:16:33.912610Z lvl=info msg="Using data dir" log_id=0_SjrEuW000 service=storage-engine service=store path=/var/lib/influxdb2/engine/data
influxdb_1 | ts=2022-03-25T21:16:33.912651Z lvl=info msg="Compaction settings" log_id=0_SjrEuW000 service=storage-engine service=store max_concurrent_compactions=2 throughput_bytes_per_second=50331648 throughput_bytes_per_second_burst=50331648
influxdb_1 | ts=2022-03-25T21:16:33.912670Z lvl=info msg="Open store (start)" log_id=0_SjrEuW000 service=storage-engine service=store op_name=tsdb_open op_event=start
influxdb_1 | ts=2022-03-25T21:16:33.942051Z lvl=info msg="index opened with 8 partitions" log_id=0_SjrEuW000 service=storage-engine index=tsi
influxdb_1 | ts=2022-03-25T21:16:33.944176Z lvl=info msg="Reading file" log_id=0_SjrEuW000 service=storage-engine engine=tsm1 service=cacheloader path=/var/lib/influxdb2/engine/wal/fd224ef0f0444b4b/autogen/2/_00001.wal size=1576906
telegraf_1 | 2022-03-25T21:16:34Z I! Starting Telegraf 1.21.4
telegraf_1 | 2022-03-25T21:16:34Z I! Using config file: /etc/telegraf/telegraf.conf
influxdb_1 | ts=2022-03-25T21:16:34.165432Z lvl=info msg="Opened shard" log_id=0_SjrEuW000 service=storage-engine service=store op_name=tsdb_open index_version=tsi1 path=/var/lib/influxdb2/engine/data/fd224ef0f0444b4b/autogen/2 duration=249.364ms
influxdb_1 | ts=2022-03-25T21:16:34.165582Z lvl=info msg="Open store (end)" log_id=0_SjrEuW000 service=storage-engine service=store op_name=tsdb_open op_event=end op_elapsed=252.912ms
influxdb_1 | ts=2022-03-25T21:16:34.165659Z lvl=info msg="Starting retention policy enforcement service" log_id=0_SjrEuW000 service=retention check_interval=30m
influxdb_1 | ts=2022-03-25T21:16:34.165687Z lvl=info msg="Starting precreation service" log_id=0_SjrEuW000 service=shard-precreation check_interval=10m advance_period=30m
influxdb_1 | ts=2022-03-25T21:16:34.165749Z lvl=info msg="Starting query controller" log_id=0_SjrEuW000 service=storage-reads concurrency_quota=1024 initial_memory_bytes_quota_per_query=9223372036854775807 memory_bytes_quota_per_query=9223372036854775807 max_memory_bytes=0 queue_size=1024
influxdb_1 | ts=2022-03-25T21:16:34.169554Z lvl=info msg="Configuring InfluxQL statement executor (zeros indicate unlimited)." log_id=0_SjrEuW000 max_select_point=0 max_select_series=0 max_select_buckets=0
influxdb_1 | ts=2022-03-25T21:16:34.476773Z lvl=info msg=Listening log_id=0_SjrEuW000 service=tcp-listener transport=http addr=:8086 port=8086
influxdb_1 | ts=2022-03-25T21:16:34.476855Z lvl=info msg=Starting log_id=0_SjrEuW000 service=telemetry interval=8h
grafana_1 | t=2022-03-25T21:16:43+0000 lvl=info msg="Request Completed" logger=context userId=1 orgId=1 uname=admin method=GET path=/api/live/ws status=0 remote_addr=172.21.0.1 time_ms=1 size=0 referer=
influxdb_1 | ts=2022-03-25T21:16:51.247533Z lvl=info msg=Unauthorized log_id=0_SjrEuW000 error="session not found"
In the /var/log/telegraf/metrics.out doesn’t seem to write anything and in telegraf.log I have:
2022-03-25T21:42:20Z D! [agent] Stopping service inputs
2022-03-25T21:42:20Z D! [inputs.mqtt_consumer] Disconnecting [tcp://eu1.cloud.thethings.network:1883]
2022-03-25T21:42:20Z D! [inputs.mqtt_consumer] Disconnected [tcp://eu1.cloud.thethings.network:1883]
2022-03-25T21:42:20Z D! [agent] Input channel closed
2022-03-25T21:42:20Z I! [agent] Hang on, flushing any cached metrics before shutdown
2022-03-25T21:42:20Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:20Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:20Z I! [agent] Stopping running outputs
2022-03-25T21:42:20Z D! [agent] Stopped Successfully
2022-03-25T21:42:28Z I! Loaded inputs: mqtt_consumer
2022-03-25T21:42:28Z I! Loaded aggregators:
2022-03-25T21:42:28Z I! Loaded processors:
2022-03-25T21:42:28Z I! Loaded outputs: file influxdb_v2
2022-03-25T21:42:28Z I! Tags enabled: host=169916e15b06
2022-03-25T21:42:28Z I! [agent] Config: Interval:30s, Quiet:false, Hostname:"169916e15b06", Flush Interval:10s
2022-03-25T21:42:28Z D! [agent] Initializing plugins
2022-03-25T21:42:28Z D! [agent] Connecting outputs
2022-03-25T21:42:28Z D! [agent] Attempting connection to [outputs.file]
2022-03-25T21:42:28Z D! [agent] Successfully connected to outputs.file
2022-03-25T21:42:28Z D! [agent] Attempting connection to [outputs.influxdb_v2]
2022-03-25T21:42:28Z D! [agent] Successfully connected to outputs.influxdb_v2
2022-03-25T21:42:28Z D! [agent] Starting service inputs
2022-03-25T21:42:28Z I! [inputs.mqtt_consumer] Connected [tcp://eu1.cloud.thethings.network:1883]
2022-03-25T21:42:38Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:38Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:48Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:48Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:58Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:42:58Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:43:08Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 10000 metrics
2022-03-25T21:43:08Z D! [outputs.file] Buffer fullness: 0 / 10000 metrics
I suppose all the connections have been done properly, however I don’t see anything in influxdb or anything comming in telegraf. TTN has data for sure, but I don’t get anything.
Some advice would be great
Just in case somebody else has the same issue, it was finally due to the fact I was using ttn v3, ie. in telegraf.conf:
topics = [ "v3/+/devices/+/up" ]
instead of:
topics = [ "+/devices/+/up" ]