Search code examples
gpudevopsprometheusgrafanapromql

Sum over unix timestamps in grafana prometheus


Lets say I have my_val for a value in prometheus, that is recorded when I use a gpu instance. I want to sum up how many hours or gpu usage I had in the past week. I can call timestamp(myval{instance="$instance"}) which will return a vector with timestamps, but I cannot call sum(idelta) over them because it is an instant vector for some reason.

Grafana also messes with the amount of data requested based on how far I am zooming.

How do I create a reliable call for every datapoint


Solution

  • Try the following query:

    count_over_time(my_val[7d]) * <scrape_interval> / 3600
    

    Where <scrape_interval> is scrape interval in seconds set in Prometheus config file for the target that exports my_val metric. This query assumes that my_val doesn't contain data points when gpu instance wasn't used.