I'm creating a plugin for Munin to monitor stats of named processes. One of the sources of information would be /proc/[pid]/io. But I have a hard time finding out what the difference is between rchar/wchar and read_bytes/written_bytes.
They are not the same, as they provide different values. What do they represent?
While the proc manpage is woefully behind (and so are most manpages/documentation on anything not relating to cookie-cutter user-space development), this stuff is fortunately documented completely in the Linux kernel source under Documentation/filesystems/proc.rst. Here are the relevant bits:
rchar
-----
I/O counter: chars read
The number of bytes which this task has caused to be read from storage. This
is simply the sum of bytes which this process passed to read() and pread().
It includes things like tty IO and it is unaffected by whether or not actual
physical disk IO was required (the read might have been satisfied from
pagecache)
wchar
-----
I/O counter: chars written
The number of bytes which this task has caused, or shall cause to be written
to disk. Similar caveats apply here as with rchar.
read_bytes
----------
I/O counter: bytes read
Attempt to count the number of bytes which this process really did cause to
be fetched from the storage layer. Done at the submit_bio() level, so it is
accurate for block-backed filesystems. <please add status regarding NFS and
CIFS at a later time>
write_bytes
-----------
I/O counter: bytes written
Attempt to count the number of bytes which this process caused to be sent to
the storage layer. This is done at page-dirtying time.