MongoDB update/insert document and Increment the matched array element

Question

I use Node.js and MongoDB with monk.js and i want to do the logging in a minimal way with one document per hour like:

final doc:

{ time: YYYY-MM-DD-HH, log: [ {action: action1, count: 1 }, {action: action2, count: 27 }, {action: action3, count: 5 } ] }

the complete document should be created by incrementing one value.

e.g someone visits a webpage first this hour and the incrementation of action1 should create the following document with a query:

{ time: YYYY-MM-DD-HH, log: [ {action: action1, count: 1} ] }

an other user in this hour visits an other webpage and document should be exteded to:

{ time: YYYY-MM-DD-HH, log: [ {action: action1, count: 1}, {action: action2, count: 1} ] }

and the values in count should be incremented on visiting the different webpages.

At the moment i create vor each action a doc:

tracking.update({ time: moment().format('YYYY-MM-DD_HH'), action: action, info: info }, { $inc: {count: 1} }, { upsert: true }, function (err){}

Is this possible with monk.js / mongodb?

EDIT: Thank you. Your solution looks clean and elegant, but it looks like my server can't handle it, or i am to nooby to make it work.

i wrote a extremly dirty solution with the action-name as key:

tracking.update({ time: time, ts: ts}, JSON.parse('{ "$inc": {"'+action+'": 1}}') , { upsert: true }, function (err) {});

Answer 1

Yes it is very possible and a well considered question. The only variation I would make on the approach is to rather calculate the "time" value as a real Date object ( Quite useful in MongoDB, and manipulative as well ) but simply "round" the values with basic date math. You could use "moment.js" for the same result, but I find the math simple.

The other main consideration here is that mixing array "push" actions with possible "updsert" document actions can be a real problem, so it is best to handle this with "multiple" update statements, where only the condition you want is going to change anything.

The best way to do that, is with MongoDB Bulk Operations.

Consider that your data comes in something like this:

{ "timestamp": 1439381722531, "action": "action1" }

Where the "timestamp" is an epoch timestamp value acurate to the millisecond. So the handling of this looks like:

 // Just adding for the listing, assuming already defined otherwise
var payload = { "timestamp": 1439381722531, "action": "action1" };

// Round to hour
var hour = new Date(
    payload.timestamp - ( payload.timestamp % ( 1000 * 60 * 60 ) )
);

// Init transaction
var bulk = db.collection.initializeOrderedBulkOp();

// Try to increment where array element exists in document
bulk.find({ 
    "time": hour,
    "log.action": payload.action
}).updateOne({
    "$inc": { "log.$.count": 1 }
});

// Try to upsert where document does not exist
bulk.find({ "time": hour }).upsert().updateOne({
    "$setOnInsert": {
        "log": [{ "action": payload.action, "count": 1 }]
    }
});

// Try to "push" where array element does not exist in matched document
bulk.find({
    "time": hour,
    "log.action": { "$ne": payload.action }
}).updateOne({
    "$push": { "log": { "action": payload.action, "count": 1 } }
});

bulk.execute();

So if you look through the logic there, then you will see that it is only ever possible for "one" of those statements to be true for any given state of the document either existing or not. Technically speaking, the statment with the "upsert" can actually match a document when it exists, however the $setOnInsert operation used makes sure that no changes are made, unless the action actually "inserts" a new document.

Since all operations are fired in "Bulk", then the only time the server is contacted is on the .execute() call. So there is only "one" request to the server and only "one" response, despite the multiple operations. It is actually "one" request.

In this way the conditions are all met:

1. Create a new document for the current period where one does not exist and insert initial data to the array.

2. Add a new item to the array where the current "action" classification does not exist and add an initial count.

3. Increment the count property of the specified action within the array upon execution of the statement.

All in all, yes posssible, and also a great idea for storage as long as the action classifications do not grow too large within a period ( 500 array elements should be used as a maximum guide ) and the updating is very efficient and self contained within a single document for each time sample.

The structure is also nice and well suited to other query and possible addtional aggregation purposes as well.