Search code examples
mysqlapimonitoringinfluxdb

InfluxDB heavy usage in monitoring


Should InfluxDB be used for monitoring networks, server status (like MySQL) and API data (e. g. Yahoo Finance)? What are the main pros versus the client software such as Wireshark?


Solution

  • InfluxDB even in the community edition (only single instance) can handle huge amount of incoming data: thousands of timeseries and millions of data values if you have sufficient storage for given amount of data. By default InfluxDB will retain incoming data forever, you can configure data retention policy for each namespace if you're interested e.g. in last 30 days.

    For monitoring MySQL have a look at Telegraf's MySQL plugin, which is a data collector that should run on MySQL server. InfluxDB is "just" a timeseries database, not data collector nor monitoring tool.

    With simple configuration (in /etc/telegraf/telegraf.conf) you can get some basic metrics:

    [[inputs.mysql]]
      servers = ["tcp(127.0.0.1:3306)/"]
    

    beside the database itself you might want to monitor system status (CPU, memory):

    [[inputs.cpu]]
      fielddrop = ["time_*"]
      percpu = false
      totalcpu = true
    [[inputs.disk]]
    [[inputs.diskio]]
    [[inputs.io]]
    [[inputs.kernel]]
    [[inputs.mem]]
    [[inputs.net]]
      interfaces = ["eth0"]
    

    Of course you're not limited to using just Telegraf for collecting metrics, you could use collectd, statsd, etc. but integration with Telegraf is probably the easiest way.

    Wireshark is a tool for packet inspection, it's completely different category of tools. Wireshark's output could be probably used for monitoring SQL queries on the fly (after doing a lot of parsing). But that kind of data are not suitable for timeseries database (you could store it in Elasticsearch or some column database).

    Timeseries database typically store metrics: number of packets, number of queries, number of connections. And aggregate them over time.