Search code examples
mongodbimportcollectionsexport

create a new collection from existing collection's object array type field in mongodb?


I would like create a new collection from an existing collection. The structure of document is like this.

{"_id" : "ObjectId('1234')",
  "field2" : "value2",
  "field3" : [
        {"field3-1": "value3", 
         "field3-2":"value4", 
         "field3-3":"value5", 
         "field3-4":"value6"},
         {"field3-1": "value7", 
         "field3-2":"value8", 
         "field3-3":"value9", 
         "field3-4":"value10"}
    ]
}

I would like a new collection that has field3-1, field3-2, field3-3 and field3-4 like the following.

{
    "source_id": "1234",
    "field3-1": "value3",
    "field3-2":"value4", 
    "field3-3":"value5", 
    "field3-4":"value6"
},
{
    "source_id": "1234",
    "field3-1": "value7", 
    "field3-2":"value8", 
    "field3-3":"value9", 
    "field3-4":"value10"
}

I tried using export by choosing only field3, it created the file like the following when I selected the type as csv, which I can't export into any collection.

"[
"{
    ""field3-1"": ""value3"",
    ""field3-2"":""value4"", 
    ""field3-3"":""value5"", 
    ""field3-4"":""value6""
}",
"{
    ""field3-1"": ""value7"",
    ""field3-2"":""value8"", 
    ""field3-3"":""value9"", 
    ""field3-4"":""value10""
}"
]"

I have almost 20 million documents in the existing collection, what could be the efficient way to achieve it? If anyone has an experience, could you give some advices?


Solution

  • Simply do an $unwind. $replaceRoot to move the sub-documents to root level. $out to a new collection.

    db.collection.aggregate([
      {
        "$unwind": "$field3"
      },
      {
        "$replaceRoot": {
          "newRoot": {
            "$mergeObjects": [
              "$field3",
              {
                // fields you want to keep
                "source_id": "$_id"
              }
            ]
          }
        }
      },
      {
        "$out": {
          "db": "output-db",
          "coll": "output-collection"
        }
      }
    ])
    

    Mongo Playground