Search code examples
grafanapromqlgrafana-templating

Grafana Cloud: dividing two queries shows "no data"

In a Grafana dashboard panel, I have two queries:

rate(container_cpu_usage_seconds_total{cloud=~"${cloud}",environment=~"${environment}",location=~"${location}",container="tlm-telemetry-service",namespace="tlm"}[5m])

and

avg(kube_pod_container_resource_limits_cpu_cores{namespace="tlm"})

which can be shown well separately: enter image description here

But when I try to use the "avg" one to divide the "rate" one:

rate(container_cpu_usage_seconds_total{cloud=~"${cloud}",environment=~"${environment}",location=~"${location}",container="tlm-telemetry-service",namespace="tlm"}[5m])/avg(kube_pod_container_resource_limits_cpu_cores{namespace="tlm"})

the result shows "no data". enter image description here

This query used to work in Grafana Version 6.7.5, but when I tried to move it to Grafana Cloud, this problem happened.

Any one has any idea?

Solution

  • This throws a No data because you have two different expression data types. On the left side, you have a range vector and the right side scalar type. The query should return the same LabelSet in each part, so you need to group the right side also with the by clause.

    E.g. the following query should work:

    sum by (namespace,container) (rate(container_cpu_usage_seconds_total{cloud=~"${cloud}",environment=~"${environment}",location=~"${location}",container="tlm-telemetry-service",namespace="tlm"}[5m])) / 
avg by (namespace,container) (kube_pod_container_resource_limits_cpu_cores{namespace="tlm"})