Search code examples
grafanainfluxdbinfluxdb-pythongrafana-alerts

Grafana alerts on bulk inserted data

I'm inserting data into InfluxDB using the Python library which inserts data in batches. From the image below what I'd like is for an alert at each of the three points the threshold is breached, but because the data is inserted in batches I can't seem to find the right combination of alert settings to do this. The best I can get is a single alert using max. last doesn't fire an alert at all because the last data point of the bulk inserts never breached the threshold.

enter image description here

Is there a way to achieve this? Maybe by checking every data point? Note the data is 1Hz and it isn't "live", it takes around an hour to collect and process the data before it is inserted into InfluxDB, but this can vary.

Solution

  • You need clever query. e.g:

    SELECT COUNT(*)
FROM <measurement>
WHERE <time-condition>
 AND value >=<threshold>

    It counts all records above thresholds - so it is single value not a time serie and it can be visualized in the graph (singlestat panel is better). You need to add appropriate time grouping, so result will be time serie, which can be visualized in the graph panel. Then create alert which will fire alarm, when result of this query is > 2.