Search code examples
monitoringdiskinfrastructurecollectdlinux-disk-free

collectd's df plugin reporting wrong free space in /home


I'm using Librato's collectd, version 5.5, in machines running Ubuntu 16.04 to collect usage metrics. However, in one of my clusters, I cannot make df plugin report all space available for / and /home.

As you can see, these mount points are in different ext4 partitions. I expected it to report both 17G and 1.7T available for / and /home respectively. Although I correctly receive the space associated to /, it reports about 4GB for /home.

$ df -h 
Filesystem Size Used Avail Use% Mounted on 
/dev/root  20G  1.4G 17G   8%   / 
devtmpfs   3.9G 0    3.9G  0%   /dev 
tmpfs      3.9G 0    3.9G  0%   /dev/shm 
tmpfs      3.9G 125M 3.8G  4%   /run 
tmpfs      5.0M 0    5.0M  0%   /run/lock 
tmpfs      3.9G 0    3.9G  0%   /sys/fs/cgroup 
/dev/sda3  1.8T 8.9G 1.7T  1%   /home 
tmpfs      789M 0    789M  0%   /run/user/0

$ cat /etc/fstab 
# <file system> <mount point> <type> <options> <dump> <pass> 
/dev/sda2       /             ext4   errors=remount-ro,relatime 0 1 
/dev/sda3       /home         ext4   defaults,relatime 1 2 
/dev/sda4       swap          swap   defaults 0 0 
proc            /proc         proc   defaults 0 0 
sysfs           /sys          sysfs  defaults 0 0 
devtmpfs        /dev          devtmpfs rw 0 0

Here is my current configuration regarding df:

<Plugin "df"> 
    MountPoint "/" 
    MountPoint "/home" 
</Plugin>

...

LoadPlugin "match_regex"

<Chain "PostCache"> 
    <Rule "ignore_df_reserved_used_metrics"> 
        <Match "regex"> 
            Plugin "^df" 
            TypeInstance "^(reserved|used)" 
        </Match> 
        Target "stop" 
    </Rule>

    # Default behavior 
    Target "write" 
</Chain>

I've tried several combinations of configurations (as well as reading the df plugin docs over and over again) and when using only FSType as below, I receive several measurements (usr, boot, bin, lib, lib64, root...), all reporting the same size of about 17G. Also, there was no measurement for home.

<Plugin df>
  FSType "ext4"
</Plugin>

I've also tried using FSType with MountPoint "/", but only root was reported then.

My last attempt was using Device in the configuration, but it didn't change anything.

<Plugin "df">
  Device "/dev/sda2"
  Device "/dev/sda3"
  MountPoint "/"
  MountPoint "/home"
  FSType "ext4"
  IgnoreSelected false
  ReportInodes false
</Plugin>

In all attempts, collectd doesn't show any error on its log:

[2017-01-13 17:06:15] Exiting normally.
[2017-01-13 17:06:15] collectd: Stopping 5 read threads.
[2017-01-13 17:06:15] collectd: Stopping 5 write threads.
[2017-01-13 17:06:15] supervised by systemd, will signal readyness
[2017-01-13 17:06:15] Initialization complete, entering read-loop.

Do you have any suggestions on what to do?


Solution

  • It turns out that probably collectd had this bug fixed in their last version, 5.7, which isn't available in Ubuntu 16.04 apt-get yet. By downloading and installing their latest nightly build, df started reporting the correct values.