Search code examples
linuxmongodbmemory-mapped-filesnosql

Why does MongoDB's memory mapped files cause programs like top to show larger numbers than normal?


I am trying to wrap my head around the internals of mongodb, and I keep reading about this

http://www.theroadtosiliconvalley.com/technology/mongodb-mongo-nosql-db/

Why does this happen?


Solution

  • So the way memorry mapped files work is that the addresses in memory are mapped byte for byte with a file on disk. This makes it really fast and but really large. Imagine a file on disk for your data taking up that size of memory.

    Why it's awesome

    In practice, this rocks because writing and reading from memory directly instead of issuing a system call (think context switch) is fast. Also, in practice, the fact that this huge memory mapped chunk doesn't fit in your physical ram is fine. Why? You only need the working set of data to fit in ram because the non-used pages are not loaded and just kept on disk. If they are needed a page fault happens and it gets loaded up. (I believe the portion that has been loaded is referred to as resident memory)

    Why it it kind of sucks

    Files mapped in memory needs to be page aligned so if you don't use up the memory space on the page boundary exactly you waste space (small tradoff)

    Summary (tldnr)

    It may look like its taking up a lot of resources because its mapping the entirety of your data to memory addresses but it doesn't really matter as that data isn't actually all being held in RAM. Mongo will pull in data as it needs it and use memory effectively to maintain a performant working set.