I know that kernel will emit detailed system memory status to kernel log when OOM Killer is triggered. Is there a way to query this information while the system is running normally?
I know that basic info can be found at /proc/meminfo but the details I cannot find is following lines in the OOM Killer output (example from my system):
Node 0 DMA: 0*4kB 0*8kB 1*16kB (U) 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15888k
B
Node 0 DMA32: 14121*4kB (UME) 18163*8kB (UME) 12588*16kB (UME) 5074*32kB (UME) 1404*64kB (UME) 484*128kB (UME) 19*256kB (UME) 0*512k
B 0*1024kB 0*2048kB 0*4096kB = 722236kB
Node 0 Normal: 399161*4kB (UME) 126769*8kB (UME) 74232*16kB (UME) 26738*32kB (UME) 5767*64kB (UME) 1079*128kB (UME) 50*256kB (UE) 2*
512kB (U) 0*1024kB 0*2048kB 0*4096kB = 5175148kB
This information tells how much free RAM the system has ready for future memory requests. I'm trying to minimize system latency and I'd like to collect and log information about how much free space I normally have in each bucket so I could better understand abnormal situation when the OOM Killer is activated because the system is often seeing high latency before that.
Is the information in the above three lines available accessible using vanilla Linux kernel without triggering OOM Killer? (Obviously, NUMA systems could have more RAM nodes so the above lines are the minimum info.)
The correct file is called /proc/buddyinfo and it looks like this:
$ cat /proc/buddyinfo
Node 0, zone DMA 0 0 1 0 2 1 1 0 1 1 3
Node 0, zone DMA32 52816 22757 16135 7177 2517 548 19 0 0 0 0
Node 0, zone Normal 27530 148696 45276 3778 6814 3711 222 0 0 0 0
The line with "zone DMA32" will be missing if the kernel is running with 32 bit kernel. See http://andorian.blogspot.com/2014/03/making-sense-of-procbuddyinfo.html for details.
The the reason why this matters is that you may end up triggering kernel OOM Killer because of memory fragmentation in case the requested memory size is bigger than the biggest available contiguous memory area. The details about memory fragmentation can be found here: http://events17.linuxfoundation.org/sites/events/files/slides/%5BELC-2015%5D-System-wide-Memory-Defragmenter.pdf