Search code examples
kubernetesdeploymentdevopsprometheusmonitoring

How to subtract two Prometheus metrics to calculate the duration of pod execution?


Im trying to get the pods runnig time from the created moment to the termination after tryin hunderd solution I think Im close I have this query gives me when pod terminated

 last_over_time(timestamp(kube_pod_status_phase{phase="Succeeded",pod=~"consumer-deployment.*"})[1h:])  

and this gives me when pod is created

last_over_time((kube_pod_created{pod=~"consumer-deployment.*"}[1h]))

but when I put - between them I get empty Empty query result What I can do ?


Solution

  • I think I found a solution this query worked and it return the runnig period for each pod

    last_over_time(timestamp(kube_pod_status_phase{phase="Succeeded",pod=~"consumer-deployment.*"})[1h:])
      - ignoring(phase) group_right()
      last_over_time((kube_pod_created{pod=~"consumer-deployment.*"}[1h:]))