Search code examples
javascriptnode.jsmongodbcapped-collections

How to use 64-bit long auto-increment counters in MongoDB


I'm implementing a logger database using MongoDB. The capped collection will contain log messages collected from several sources across the network. Since I want to do $lte/$gte queries on _id afterwards I need to have an _id that grows as a monotonic function.

To achieve that I've implemented the auto-incremented counter described in this article http://docs.mongodb.org/manual/tutorial/create-an-auto-incrementing-field/

My code looks like that:

var mongo = require("mongodb");
var Promise = require('es6-promise').Promise;

function connectToDB(mongo, uri) {
    return new Promise(function(resolve, reject) {
        mongo.MongoClient.connect(uri, function (err, db) {
            if(err) reject(err);
            else resolve(db);
        });
    });
}

function getNextSequenceNumber(db, counterName) {
    return new Promise(function(resolve, reject) {
        db.collection("counters", function (err, collection) {
            if (err) reject(err);
            else {
                var criteria = { _id: counterName };
                var sort =  {$natural: 1};
                var update =  { $inc: { seq: 1 } };
                var options = {remove: false, new: true, upsert: true};
                collection.findAndModify(criteria, sort, update, options, function(err, res) {
                    if(err) reject(err);
                    else resolve(res.seq);
                });
            }
        });
    });
}

It works perfectly fine, but I've read that by default the number fields used in MongoDB are actually floats. The problem is that my database is a capped collection of log entries and it is going to have lots of entries. Moreover, since this is a capped collection the old entries will be overwritten but the counter will keep growing. Having counter as a float I cannot guarantee the system will keep on working after a few years. My question is how can I force MongoDB to use 64 bit counter in this particular case.

Please provide some code examples.


Solution

  • MongoDB (or rather BSON) has the NumberLong type, which is a 64-bit signed integer.

    From Node.js you can use it in your update statement to create the seq property of that type:

    var update =  { $inc : { seq : mongo.Long(1) } };
    

    This also seems to convert the seq property of existing documents to NumberLong.