Search code examples
mongodbmongodumpmongorestore

How to copy 5TB mongodb to a new machine within LAN

I've tried:

  1. SCP the (custom) DB directory from old machine to new machine - didn't work. Mongo complained of corrupt wiredTiger files

  2. Ran mongodump and mongorestore - took about 5 times as long. The process was interrupted by someone who thought it'd finished, so missed a small amount of data

  3. Realised 1. didn't work due to mongo not shutting down properly first.

Tried again, results in the following error:

2017-05-20T17:12:04.407+0100 E STORAGE [initandlisten] WiredTiger (24) [1495296724:407111][46215:0x7f7bf304bb80], file:collection-299--5136948165504185516.wt, session.open_cursor: /home/myuser/mymongodir/data/db//collection-299--5136948165504185516.wt:

Too many open files 2017-05-20T17:12:04.407+0100 I - [initandlisten] Invariant failure: ret resulted in status UnknownError 24: Too many open files at src/mongo/db/storage/wiredtiger/wiredtiger_session_cache.cpp 69`

Solution

  • You should try to run:

    ulimit -a

    It will gives you the current open limit, you can then increase it accordingly using the -n option, which will increase the limit of your open files, allowing mongo to run safely.