Search code examples
node.jsmongodbbulkupdate

Mongo Bulk Updates - which succeeded (matched and modified) and which did not?


In order to improve the performance of many single Mongo document updates @Node, I consider using Mongo's Bulk operation - to update as many as 1000 documents at each iteration.

In this use case, each individual update opeartion may or may not occur - an update will only occur if the document version had not changed since it was last read by the updater. If a docuemnt was not updated, the application needs to retry and/or do other stuff to hadnle the situation.

Currently the Node code looks like this:

  col.update(
        {_id: someid, version:someversion},
        {$set:{stuf:toupdate, version:(someversion+1)}},
        function(err, res) {
           if (err) {   
              console.log('wow, something is seriously wrong!');
              // do something about it...

              return;
           }
           if (!res || !res.result || !res.result.nModified) { // no update
              console.log('oops, seems like someone updated doc before me);
              // do something about it...

              return; 
           }
           // Great! - Document was updated, continue as usual...
  });

Using Mongo's Bulk unordered operations, is there a way to know which of the group of (1000) updates had succeeded and which had not been performed (in this case due to wrong version)?

The code I started playing with looks like:

var bulk = col.initializeUnorderedBulkOp();
bulk.find({_id: someid1, version:someversion1}).updateOne(
                 {$set:{stuf:toupdate1, version:(someversion1+1)}});
bulk.find({_id: someid2, version:someversion2}).updateOne(
                 {$set:{stuf:toupdate2, version:(someversion2+1)}});
...
bulk.find({_id: someid1000, version:someversion1000}).updateOne(
                 {$set:{stuf:toupdate1000, version:(someversion1000+1)}});
bulk.execute(function(err, result) {
           if (err) {   
              console.log('wow, something is seriously wrong!');
              // do something about it...

              return;
           }
           if (result.nMatched < 1000) { // not all got updated
              console.log('oops, seems like someone updated at least one doc before me);
              // But which of the 1000 got updated OK and which had not!!!!

              return; 
           }
           // Great! - All 1000 documents got updated, continue as usual...

});


Solution

  • I was unable to find a Mongo solution for that.

    The solution I used was to revert to per document operation if the bulk operation failed... This gives reasonable performance in most cases.