MongoDB - $merge that works in Compass but not from a Node.js Lambda

Question

I have an aggregation pipeline that works fine in Compass but when I use it from a NodeJS Lambda it doesn't do anything. There is no error, and certainly no connection issue or anything like that because I'm also inserting data and creating indexes on the fly with the same connection details. Why would this not work?

Lambda code:

// 'db' obtained from connection pooling elsewhere

await db.collection(collectionName).aggregate(aggregationStatement);

Aggregation statement:

[
    {
        "$match": {
            "type": "costForecast"
        }
    },
    {
        "$sort": {
            "when": 1
        }
    },
    {
        "$group": {
            "_id": {
                "type": "$type"
            },
            "when": {
                "$last": "$when"
            },
            "type": {
                "$last": "$type"
            },
            "serviceId": {
                "$last": "$serviceId"
            },
            "serviceName": {
                "$last": "$serviceName"
            },
            "data": {
                "$last": "$data"
            }
        }
    },
    {
        "$project": {
            "when": 1,
            "type": 1,
            "serviceId": 1,
            "serviceName": 1,
            "data": "$data.ForecastResultsByTime"
        }
    },
    {
        "$merge": {
            "into": "CostsOut",
            "on": [ "type", "serviceId" ],
            "whenMatched": "replace",
            "whenNotMatched": "insert"
        }
    }
]

Needless-to-say all the fields are present and the destination collection for the $merge contains the unique index to support the "on" declaration. The function I am trying to achieve is to find the latest example of a "costForecast" item in the original collection, pick off a few fields and slightly flatten the data, and then insert or update the "CostsOut" collection with the document taking "type" and "serviceId" as a unique compound index.

Many thanks for your help.

Answer 1

.aggregate() is not awaitable, it just returns a cursor and not a promise.

To tell Mongo, that it should execute the aggregation you must call .toArray() or .forEach()

await db.collection(collectionName).aggregate(aggregationStatement).toArray();