Search code examples
mongodbmongoimport

How to import large json file to mongodb?


I am trying to import a json file which is 170MB in size. I am using mongoimport --db test --collection nhentai --file ~/py/nhentai.json --jsonArray. It import exactly 23000 documents and gives the following message.

2020-12-06T08:57:22.763+0000    connected to: mongodb://localhost/
2020-12-06T08:57:23.515+0000    Failed: cannot decode 32-bit integer into a slice
2020-12-06T08:57:23.517+0000    23000 document(s) imported successfully. 0 document(s) failed to import.

There's error on the second line. I don't understand the error and I couldn't find anything upon googling. I tried to split the first 23000 objects from json array and place it in another file and it worked without any errors. How can I import the entire file in one go?


Solution

  • For those who are looking for the answer to this question. I looked into many ways of doing it but couldn't find any answers. I then tried doing the same with Mongodb Compass and it worked just fine. Mongodb compass is a neet utility to take a look at your database and it has bunch of other functionalities as well.