I am trying to list and sort all running processes in Windows by their memory consumption with the python psutil module. However when I query a processes memory_info attribute none of the various metrics will get above 4 gigabytes. Why is this limited and how can I get around this?
I did try using proc.memory_full_info() which would theoretically fix this issue with the uss memory metric, however I cannot figure out how to do this without causing an AccessDenied error.
sample script:
import psutil
from psutil._common import bytes2human
procs = [p.info for p in psutil.process_iter(attrs=['memory_info', 'memory_percent', 'name'])]
for proc in sorted(procs, key=lambda p: p['memory_percent'], reverse=True):
print("{} - {}".format(proc['name'], bytes2human(getattr(proc['memory_info'], 'rss'))))
I'm open to any other way of profiling memory usage, psutil just seems to be the best tool I found out there.
You are using 32bit Python. By default a 32bit process has a 4GiB memory limit. On a software level, that limit comes from the underlying C types limits (and psutil's core is written in C).
For example, almost all [MS.Docs]: PROCESS_MEMORY_COUNTERS structure members have the SIZE_T type.
Example:
32bit:
>>> import sys, ctypes >>> from ctypes import wintypes >>> sys.version '3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)]' >>> >>> ctypes.sizeof(wintypes.DWORD), ctypes.sizeof(ctypes.c_size_t), ctypes.sizeof(wintypes.HANDLE) (4, 4, 4)
64bit:
>>> import sys, ctypes >>> from ctypes import wintypes >>> sys.version '3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)]' >>> >>> ctypes.sizeof(wintypes.DWORD), ctypes.sizeof(ctypes.c_size_t), ctypes.sizeof(wintypes.HANDLE) (4, 8, 8)
For more details on how to distinguish between different CPU architectures Python processes, check: [SO]: How do I determine if my python shell is executing in 32bit or 64bit mode on OS X? (@CristiFati's answer)