Python version: 2.7
I am trying to normalize below json data avaialble in MongoDB . for this i am using json_normalize. However I am not able to flatten the data within "stages" element. It is coming within collection. When I process this "stage" in a row wise it is working fine. How to do I flatten the content within stage along with rest of the data? Sample data is as below.
{
"_id" : ObjectId("5a8ffe8aaacee33cd21aabcb"),
"niyoCustomerCompanyId" : 210068,
"version" : 2,
"versionDate" : ISODate("2018-02-23T17:14:10.614+05:30"),
"data" : {
"CustomerCompanyId" : 210068,
"companyName" : "XYZ Tractors ",
"mobileNo" : "***22****",
"cif" : "25",
"typeId" : 3,
"PartnerCompanyId" : "163929",
"companyDetails" : {
"sectorType" : "30",
"sectorSubType" : "2",
"pincode" : "431602",
"email" : "[email protected]",
"stages" : [
{
"name" : "Prequalification",
"status" : "done"
},
{
"name" : "Prequalification1",
"status" : "pending"
}
],
"currentStage" : "Pre-Qualification"
}
}
}
Code I tried with is as below.
cursor_only_data_stage = customer_collection.find({})
stage_data = list(cursor_only_data_stage)
stage_data_df_1 = pd.DataFrame(json_normalize(stage_data))
Expected Output is as below.
id cif stage status version versiondate
5a8ffe8aaacee33cd21aabcb 25 Prequalification done 2 2018-02-23T17:14:10.614+05:30
5a8ffe8aaacee33cd21aabcb 25 Prequalification1 pending 2 2018-02-23T17:14:10.614+05:30
I was able to do the required
all_stage_data_flat = pd.DataFrame(json_normalize(all_stage_data_list, [['data','companyDetails', 'stages']],
['_id','version','versionDate',['data','cif']]))