How can I get the sub document fields at the top level using aggregation without using project in mongodb?

Question

I have a mongodb collection having one sub document payload which has variable number of fields.Sample two mongo documents as below

{'key1': 'value1',
 'key2': 'value2',
 'payload': { 'pkey1':'pvalue1',
              'pkey2':'pvalue2',   
            }    
}
{'key1': 'value1',
 'key2': 'value2',
 'payload': { 'pkey1':'pvalue1',
              'pkey2':'pvalue2',
              'pkey3':'pvalue3',

            }    
 }

I would like the output to be as below :`

{'key1': 'value1',
 'key2': 'value2',
 'pkey1':'pvalue1',
 'pkey2':'pvalue2'     
}
{'key1': 'value1',
 'key2': 'value2',
 'pkey1':'pvalue1',
 'pkey2':'pvalue2',
 'pkey3':'pvalue3'
}

The reason I don't want to use project is I don't know how many fields are present in the payload sub document.I want to store the result of the aggregate to the other collection.I was thinking to use the for each however data is huge So it will take lot of time to complete the operation. Please advise.

Answer 1

Currently there is no way to do that in aggregation, you need to know the fields in advance. But this feature it might be introduce in the next releases.

You need to do a for each, and reshape to new docs, into a new collection:

db.test.find().forEach(function(doc) {
    var new_doc = {};
    for (var key in doc) {

        if (key != 'payload'){
            new_doc[key] = doc[key] 
        }
        else{
                for (var key in doc.payload) {
                    new_doc[key] = doc.payload[key]
                };  
        }
    };  
    db.test2.save(
        new_doc
    );
});