I'm using mmap syscall to load file memory for reading by several other processes (with flag MAP_SHARED and MAP_POPULATE to load all memory in advance.).
Checking the process memory consumption seems to ignore the fact that the mapped file memory is shared. Each process memory usage contains the entire mapped file in memory according to ps -aux command.
Is there a way to differentiate between shared and private process memory ?
Even if the memory is physically shared, each process has it's own virtual memory mapping and will "consume" an independent addressing space even if mapping the same area - you can see it in the VSIZE column. In any case (even after a malloc()) the kernel will reserve an independent addressing space for each process, but will not immediately allocate the equivalent pages in the backing store.
As soon as you start reading/writing to the allocated memory, virtual pages are being assigned to the corresponding backing store: the resident set increases correspondingly.
These pages are "clean" if you just read, "dirty" if you write to them, until they are not synchronized. (see /proc/PID/smaps, /proc documentation, man pmap: this tool let you differentiate shared and private memory, dirty and clean pages)
MAP_POPULATE option pre-assign all the pages immediately. If the mapping is MAP_PRIVATE the pages are allowed to be kept dirty, until they are not explicitly synchronized with msync(). Dirty pages are cached in RAM, and in fact the available system memory decrease as soon as you use more pages, with private mappings.
On the other side, using MAP_SHARED, you require to the kernel to keep the virtual pages synchronized (ie, the pages must be clean, as soon as possible, even if asynchronously) with the backing store.
But with a mapped file the backing store is not memory, but it's the file - as it was swap space. So you have the odd effect that the mapped file is counted as resident set.
This can be verified observing the content of /proc/PID/status:
VmRSS size of memory portions. It contains the three
following parts (VmRSS = RssAnon + RssFile + RssShmem)
RssAnon size of resident anonymous memory
RssFile size of resident file mappings
RssShmem size of resident shmem memory (includes SysV shm,
mapping of tmpfs and shared anonymous mappings)
VmRSS is what is usually reported as RSS.
RSS is not counting physical memory in this case, but the file mapping: the total system memory is mostly unaffected when requesting MAP_SHARED, since mostly no pages are cached (cached, dirty pages would be accounted in RssAnon). They must be flushed to the shared backing store (the file) as soon as possible.
As you correctly observed, RSS is counted once per process, even if shared. You can find in /proc/PID/smaps another counter, PSS (proportional set size) - see this answer too:
What does pss mean in /proc/pid/smaps
The "proportional set size" (PSS) of a process is the count of pages it has in memory, where each page is divided by the number of processes sharing it. So if a process has 1000 pages all to itself, and 1000 shared with one other process, its PSS will be 1500
One note: You can have shared memory using MAP_SHARED|MAP_POPULATE only using tmpfs - otherwise you will be using a file-based sharing. If you are looking for IPC shared memory, have a look to shmget().
I've verified my statements with a tiny program, which mapped 2 GB from a file with different flags, and then wrote 1/3 of the pages with memset().
ps -u output:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
xxxx 15767 0.5 4.1 2004216 666944 pts/1 S+ 12:54 0:00 ./mm 1
from /proc/PID/smaps:
7f143e5ec000-7f14b870c000 rw-p 00000000 fd:00 201562609 /home/xxxx/test/mm.dat
Size: 2000000 kB
Rss: 666668 kB
Pss: 666668 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 666668 kB
Referenced: 666668 kB
Anonymous: 666668 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd wr mr mw me ac sd
from /proc/PID/status:
VmRSS: 666944 kB
RssAnon: 666560 kB
RssFile: 384 kB
RssShmem: 0 kB
ps -u output:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
xxxx 17045 31.0 12.3 2004216 2000412 pts/1 S+ 13:01 0:01 ./mm 2
from /proc/PID/smaps:
7f14aa983000-7f1524aa3000 rw-p 00000000 fd:00 201562609 /home/xxxx/test/mm.dat
Size: 2000000 kB
Rss: 2000000 kB
Pss: 2000000 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 2000000 kB
Referenced: 2000000 kB
Anonymous: 2000000 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd wr mr mw me ac sd
from /proc/PID/status:
VmRSS: 2000412 kB
RssAnon: 2000032 kB
RssFile: 380 kB
RssShmem: 0 kB
ps -u output:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
xxxx 17670 18.0 4.1 2004216 666944 pts/1 S+ 13:06 0:00 ./mm 3
from /proc/PID/smaps:
7f01db018000-7f0255138000 rw-s 00000000 fd:00 201562609 /home/xxxx/test/mm.dat
Size: 2000000 kB
Rss: 666668 kB
Pss: 666668 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 666668 kB
Private_Dirty: 0 kB
Referenced: 666668 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd wr sh mr mw me ms sd
from /proc/PID/status:
VmRSS: 666944 kB
RssAnon: 96 kB
RssFile: 666848 kB
RssShmem: 0 kB
ps -u output:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
xxxx 18259 16.1 12.3 2004216 2000480 pts/1 S+ 13:10 0:00 ./mm 4
from /proc/PID/smaps:
7ff0d020b000-7ff14a32b000 rw-s 00000000 fd:00 201562609 /home/xxxx/test/mm.dat
Size: 2000000 kB
Rss: 2000000 kB
Pss: 2000000 kB
Shared_Clean: 0 kB
Shared_Dirty: 0 kB
Private_Clean: 2000000 kB
Private_Dirty: 0 kB
Referenced: 2000000 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd wr sh mr mw me ms sd
from /proc/PID/status:
VmRSS: 2000480 kB
RssAnon: 96 kB
RssFile: 2000384 kB
RssShmem: 0 kB
ps -u output:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
xxxx 18259 0.1 12.3 2004216 2000480 pts/1 S 13:10 0:00 ./mm 4
xxxx 19521 5.8 12.3 2004216 2000480 pts/1 S+ 13:19 0:00 ./mm 4
from /proc/PID/smaps, one first PID only:
7ff0d020b000-7ff14a32b000 rw-s 00000000 fd:00 201562609 /home/xxxx/test/mm.dat
Size: 2000000 kB
Rss: 2000000 kB
Pss: 1000000 kB
Shared_Clean: 2000000 kB
Shared_Dirty: 0 kB
Private_Clean: 0 kB
Private_Dirty: 0 kB
Referenced: 2000000 kB
Anonymous: 0 kB
AnonHugePages: 0 kB
Swap: 0 kB
KernelPageSize: 4 kB
MMUPageSize: 4 kB
Locked: 0 kB
VmFlags: rd wr sh mr mw me ms sd
from /proc/PID/status:
VmRSS: 2000480 kB
RssAnon: 96 kB
RssFile: 2000384 kB
RssShmem: 0 kB