Search code examples
node.jsmongodbtoarray

MongoDB toArray performance


I'm trying to build a category tree from a term collection in Mongo/Node, but first I select all tree elements using $in:

console.time('termsCol.find');
var terms = await termsCol.find({term_id: {'$in': flatTree}});
console.timeEnd('termsCol.find');

console.time('termsCol.toArray');
terms = await terms.toArray();
console.timeEnd('termsCol.toArray');

This performs:

termsCol.find: 0.162ms
termsCol.toArray: 30.910ms

I've read posts about toArray performance on SO, but would like to know if anything has changed, because this takes most of my time during page request.
I have an index on that collection and it returns 300 terms within ~0.15ms, but that doesn't help me when I have to wait another 30ms to use that data further down in js.
If there's no way to improve this toArray business, I'll probably create a cache collection and store the complete term trees there (hope they fit 16MB).


Solution

  • reference http://mongodb.github.io/node-mongodb-native/2.0/tutorials/streams/ you can stream results one by one and can make array of ids.

    var MongoClient = require('mongodb').MongoClient
    var url = 'mongodb://localhost:27017/myproject';
    // Use connect method to connect to the Server
    MongoClient.connect(url, function(err, db) {
        console.log("Connected correctly to server");
     var col = db.collection('terms');
        var ids = []
        var findCursor = col.find({term_id: {'$in': flatTree}});
        findCursor.on("data", function(data) {
           ids.push(data._id)
        });
        findCursor.on("end", function(data) {
          // let's finish
          console.log(ids)
        }); 
    }); 
    

    i didn't check time but sure it should less then (termsCol.find: 0.162ms + termsCol.toArray: 30.910ms)