I am running Prometheus Node Exporter inside of a Docker container and want it to report filesystem space usage information for filesystems on the Docker container's host.
For most of my filesystems, most filesystem metrics are absent except for the node_filesystem_device_error metric that signifies that metric collection has failed.
This could be solved by running the node_exporter binary directly on the host, but I want to run my entire observability infrastructure via Docker.
To emit host filesystem metrics from within a node-exporter Docker container, the container needs to bind-mount the host's / path to a container path like /rootfs. Then, node-exporter needs to be started with the command line argument --path.rootfs=/rootfs so it knows where to find the filesystems.
A complete Docker Compose / Docker Swarm configuration for Prometheus node-exporter is below. It is also configured to report statistics about the host's networking configuration.
version: "3.9"
services:
node_exporter:
image: prom/node-exporter:v1.0.1
ports:
- target: 9100
published: 9100
protocol: tcp
mode: host
volumes:
# Remember to use read-only bind mounts.
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /:/rootfs:ro
command:
- "--web.listen-address=:9100"
- "--path.procfs=/host/proc"
- "--path.sysfs=/host/sys"
- "--path.rootfs=/rootfs" # Necessary for collecting host filesystem metrics.
- "--collector.filesystem.ignored-mount-points='^/(sys|proc|dev|host|etc|rootfs/var/lib/docker/containers|rootfs/var/lib/docker/overlay2|rootfs/run/docker/netns|rootfs/var/lib/docker/aufs)($$|/)'"
deploy:
mode: global
# This listens to port 9100 ON THE HOST.
# This container does not have its own IP address.
# Binding to the host is necessary for node-exporter to collect accurate
# networking statistics about the host.
networks:
default:
external: true
name: host