Search code examples
dockerprometheuspromqlcadvisorvictoriametrics

cadvisor machine_cpu_cores metrics comes every 5 minutes

I am using Victoriametrics, cadvisor and docker to find our per container cpu usage. I am running the following query against cadvisor metrics

(sum(rate(container_cpu_usage_seconds_total{name != ''}[1m])) BY (instance, name) * 100) / ignoring(name) (sum by (instance) (machine_cpu_cores))

This should divide cpu_usage by machine_cpu_cores, but the metric machine_cpu_cores is available every 5 minutes, inspite of having a shorter collection interval in my config

global:
  scrape_interval: 10s

scrape_configs:
  - job_name: node_exporter
    static_configs:
      - targets: ["localhost:9100"]

  - job_name: cadvisor
    static_configs:
      - targets: ["localhost:8080"]

This leads to the following break in the graph

enter image description here

This metric should be available at at least a 1 minute resolution. Has anyone else faced these issues with cadvisor?

Solution

  • I was able to achieve this by using the on (instance) group_left() expression instead of the ignoring flag

    (sum(rate(container_cpu_usage_seconds_total{name != ''}[1m])) BY (instance, name) * 100) / on(instance) group_left() (sum by (instance) (machine_cpu_cores))