Search code examples
apache-pig

pig, turn an array into multiple records


I have an array in my record:

{  
   "resource":"rest-api-v1",
   "accessControlList":[  
      {  
         "methods":{  
            "methodTypes":[  
               "DELETE"
            ]
         },
         "Users":[  
            "user2"
         ]
      },
      {  
         "methods":{  
            "methodTypes":[  
               "CREATE"
            ]
         },
         "Users":[  
            "user1",
            "user2"
         ]
      }
   ]
}

in accessControlList array, there are 2 elements in the array. How can i turn this 1 record into 2? I want the result looks like:

resource: rest-api-v1
accessControl:
{  
             "methods":{  
                "methodTypes":[  
                   "DELETE"
                ]
             },
             "Users":[  
                "user2"
             ]
}

And

resource: rest-api-v1
accessControl:
{  
             "methods":{  
                "methodTypes":[  
                   "DELETE"
                ]
             },
             "Users":[  
                "user2"
             ]
}

In hive i can do a LATERAL VIEW EXPLODE(), but in pig i don't know how.


Solution

  • The pig FLATTEN operator does what you're looking for: https://pig.apache.org/docs/r0.16.0/basic.html#flatten