Search code examples
mongodbserverwindows-serviceswindows-server-2012production-environment

Memory issues with new version of Mongodb

I'm using Mongodb on my Windows server 2012 for more than two years. Since the last update some weird issues started to happen which in the end lead to usage of the entire RAM memory.

The service Iv'e configured for Mongodb is as follows:

logpath=d:\data\log\mongod.log 
dbpath=d:\data\db
storageEngine=wiredTiger
rest=true

#override port
port=27017

#configsvr = true
shardsvr = true

And in order to limit the Cache memory usage Iv'e added the following line:

wiredTigerCacheSizeGB=10

And this is where the weird stuff started happening. When I check the task manager it says that now Mongodb is really limited to 10GB as I defined in the service but it is actually using a lot more than 10GB.

In the first image you can see the memory consumption sorted by RAM consumption

enter image description here

While in fact the machine I'm using has 28GB in total enter image description here

This crazy consumption leads to failure in the scripts I'm running, even the most basic ones, even when I only run simple queries like 'count' or 'distinct', I believe that this is a direct results of the memory consumption.

When I checked the log files I saw that there are many open connections that even when the session ends it indicates that still the same amount of connections is opened:

enter image description here

So in the end I have two major questions: 1. Is there a way of solving this issue without downgrading the Mongodb version? 2. The config file looks right? is everything there is necessary?

Solution

  • Memory usage in WiredTiger is a two-level cache:

    • First is the WiredTiger cache as controlled by --wiredTigerCacheSizeGB
    • Second is the Operating System filesystem cache. MongoDB automatically uses all free memory that is not used by the WiredTiger cache or by other processes

    See also WiredTiger memory usage

    For OS filesystem cache, MongoDB doesn't manage the memory it uses directly - it lets the OS manage it. Windows will try to use every last scrap of physical memory if it can - but lots of it should and will be thrown out if other processes request memory.

    An alternative is to run mongod in a container (e.g. lxc, cgroups, Docker, etc.) that does not have access to all of the RAM available in a system.

    Having said the above:

    • You are also running another database in the server i.e. mysqld. MongoDB, like some databases will perform better on a dedicated server to reduce memory contention.

    • Task Manager shows mongod is using 10GB, although the machine is using up to ~28GB. This may or may not be mongod as you have other processes as well.

    Useful resources: