Search code examples
kubernetesgrafanapromql

Host's pods panel in Grafana

I want to have a panel in Grafana which displays what pods are currently running in a host. For the host variable I have the following query (the job variable is just label_values(node_uname_info, job).):

label_values(node_uname_info{job="$job"}, instance)

This gives me an array of sockets: host_ip:port

I can get the pod names from kube_pod_info{job="$job", host_ip="$host_ip"}, but in order to get the IP I need to remove the port part of the socket:

label_replace(node_uname_info{job="$job", instance="$node"}, "host_ip", "$1", "instance", "(.*):.*")

I haven't found how to use the new host_ip label in the pod query to eventually get all the pod label values of kube_pod_info. I don't want to put the label_replace in Prometheus to avoid data duplication - is there a way to use the new host_ip label in the pod query?

Edit:

I added the host_ip variable with the regex as shan1024 showed in his answer and changed the panel's query to:

sum by (pod) (kube_pod_info{job="$job", host_ip="$host_ip"})

Then I changed the panel's visualization to table and added column styles to Time and Value (chose type Hidden). This allows me to display the host's running pods in a list-like fashion.

Solution

  • This is actually quite easy to do in Grafana and no need to change labels in Prometheus. You just need to add a regex in the instance variable (when we add a regex with a capturing group, the value(s) of the 1st captured group will be the value(s) of the variable).

    e.g.

    Variable definition without Regex (you get host_ip:port)-

    enter image description here

    Variable definition with Regex (you only get host_ip)-

    enter image description here

    Then you can add a new variable with value kube_pod_info{ host_ip="$instance" } to get all pods in the selected host.