Search code examples
javascriptnode.jsmongodbnodejitsu

Procedural 2d-array mongoDB insert on node.js runs out of memory

I am trying to insert a 2000 x 2000 2d array into my mongoDB instance via my node.js server.

Right now the way i'm doing it is: 

for (i=0; i<2000; i++){
            //i = height of the tile map
            for(x=0; x<2000; x++){
                //x is the width of the tile map
                driver.mongo.insert('worldTiles', {loc: [x,i], elevation: obj.server.theArray[i][x]}, function (err,results){
                    if(!err){

                    }else{
                        console.log('Ran into an error inserting into worldTiles on Mongo');
                    }
                })
            }
            console.log(i);
        }

Now, the problem I am facing is that at around line 576 my node.js instance slows down dramatically (one line processed per 40 seconds or so) and then runs out of memory. I don't understand how/why this is happening.

Am I approaching this problem correctly? Any ideas on how I can get this 2d array into my database without floundering?

Thanks for your help all.

Solution

  • It looks like the problem is with the provider of my mongoDB instance. I just found out that when spooling a mongo instance on jitsu there is a ~64MB limit on the size of the memory usage. Thus, at exactly that point, I got the "out of memory" error.

    Wish they would somehow make the error message a bit more descriptive including what device was running out of memory... rolls eyes.

    Case closed.