Does anyone know how to get a sorted list of pods on a given node based on their disk space consumption?
The command below helps me in listing pods based on a given node but my requirement is to list those pods which are causing high disk space usage as part of investigating and solving the DiskPressure eviction state.
kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=<NODE_NAME>
I was able to find commands to list CPU and memory usage of nodes (ref: https://github.com/kubernetes/kubernetes/issues/17512#issuecomment-317757388) but nothing related to node disk usage.
alias util='kubectl get nodes --no-headers | awk '\''{print $1}'\'' | xargs -I {} sh -c '\''echo {} ; kubectl describe node {} | grep Allocated -A 5 | grep -ve Event -ve Allocated -ve percent -ve -- ; echo '\'''
# Get CPU request total (we x20 because because each m3.large has 2 vcpus (2000m) )
alias cpualloc='util | grep % | awk '\''{print $1}'\'' | awk '\''{ sum += $1 } END { if (NR > 0) { print sum/(NR*20), "%\n" } }'\'''
# Get mem request total (we multiply by 75 because because each m3.large has 7.5G ram )
alias memalloc='util | grep % | awk '\''{print $5}'\'' | awk '\''{ sum += $1 } END { if (NR > 0) { print sum/(NR*75), "%\n" } }'\'''
found an answer to my requirement - list pods disk usage running in a given node
kubectl get --raw /api/v1/nodes/<NODE_NAME>/proxy/stats/summary | jq '.pods[0] | "PodName: ", .podRef.name, "usedBytes: ", .containers[].rootfs.usedBytes'
get node filesystem usage
kubectl get --raw /api/v1/nodes/<NODE_NAME>/proxy/stats/summary | jq '.node.fs.usedBytes'
get node imageFs usage
kubectl get --raw /api/v1/nodes/<NODE_NAME>/proxy/stats/summary | jq '.node.runtime.imageFs.usedBytes'
get node iNodes stats
kubectl get --raw /api/v1/nodes/<NODE_NAME>/proxy/stats/summary | jq '.node.fs.inodesFree'
kubectl get --raw /api/v1/nodes/<NODE_NAME>/proxy/stats/summary | jq '.node.runtime.imageFs.inodesFree'