Search code examples
mongodbaggregation-frameworkchangestream

MongoDB Change Stream Array Truncation


The mongo documentation on update change events says that the update description will have an array of removed fields, a document of update fields and an array of truncated arrays. The removed and updated fields are pretty straight forward, but I'm having trouble understanding what the truncated arrays are.

The documentation says

An array of documents which record array truncations performed with pipeline-based updates using one or more of the following stages:

  • $addFields
  • $set
  • $replaceRoot
  • $replaceWith

But try as I might, I can't seem to figure out how to even cause an update event that includes truncated arrays.

Any help understanding what this field is for and / or an example of how to cause an update that includes it would be greatly appreciated.


Solution

  • I did not know that the change stream document had truncatedArrays field. So, I tried to set up the change stream in MongoDB version 4 and 5.

    MongoDB Enterprise rs0:PRIMARY> db.coll.find();
    { "_id" : ObjectId("63b2d783"), "a" : 1, "b" : [ { "c" : 1, "d" : "qwq" }, { "c" : 2, "d" : "mlo" } ] }
    { "_id" : ObjectId("63b2d784"), "a" : 2, "b" : [ { "c" : 4, "d" : "hyt" }, { "c" : 5, "d" : "nhw" } ] }
    

    In another window,

    MongoDB Enterprise rs0:PRIMARY> cs = db.coll.watch([], {"fullDocument": "updateLookup"});
    MongoDB Enterprise rs0:PRIMARY> while(!cs.isExhausted()){
    ... if(cs.hasNext()){
    ...     print(tojson(cs.next()));
    ...     }
    ... }
    

    Then I ran an update.

    MongoDB Enterprise rs0:PRIMARY> db.coll.update({},{$set:{"a":3}})
    WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
    

    There was no such field in the change stream.

    {
            "_id" : {"_data" : "8263B2D474000000012B022C0100296E5A100439C39"},
            "operationType" : "update",
            "clusterTime" : Timestamp(1672664180, 1),
            "fullDocument" : {
                    "_id" : ObjectId("63b2d783"),
                    "a" : 3,
                    "b" : [
                            {"c" : 1,"d" : "qwq"},
                            {"c" : 2,"d" : "mlo"}
                    ]
            },
            "ns" : {
                    "db" : "test","coll" : "coll"
            },
            "documentKey" : {"_id" : ObjectId("63b2d783")},
            "updateDescription" : {
                    "updatedFields" : {
                            "a" : 3
                    },
                    "removedFields" : [ ]
            }
    }
    

    Next, I updated the server to version 6 and executed this update query to slice the array.

    db.coll.update(
    {},
    [
        {$set: {"b": {$slice: ["$b",1]}}}
    ]
    );
    

    And, there it was. Showing the array field name and it's new size.

    {
            "_id" : {"_data" : "8263B2D756000000012B022C0100296E5A1004A7FD82"},
            "operationType" : "update",
            "clusterTime" : Timestamp(1672664918, 1),
            "wallTime" : ISODate("2023-01-02T13:08:38.584Z"),
            "fullDocument" : {
                    "_id" : ObjectId("63b2d1d6"),
                    "a" : 1,
                    "b" : [
                            {"c" : 1,"d" : "qwq"}
                    ]
            },
            "ns" : {
                    "db" : "test","coll" : "coll"
            },
            "documentKey" : {"_id" : ObjectId("63b2d1d6")},
            "updateDescription" : {
                    "updatedFields" : {
    
                    },
                    "removedFields" : [ ],
                    "truncatedArrays" : [
                            {
                                    "field" : "b",
                                    "newSize" : 1
                            }
                    ]
            }
    }
    

    I couldn't find any other way to cause this using $replaceRoot/$replaceWith.