Search code examples
jsonamazon-web-servicesaws-glue

Glue Crawler to read Json Array

I have a Json array file of the format:- [[{Key1:Value1},{Key2:Value2},{Key3:Value3}],[{Key1:Value4},{Key2:Value5},{Key3:Value6}]]

I need to crawl the above file using AWS glue and read the json schema with each key as a column in the schema. I tried using the standard json classifier but it does not seem to work and the schema loads as an array. I need to read the json file from S3 and load it in an RDS Database. Any suggestions on how to make the transformation either through glue job or classifiers would be helpful.

Solution

  • Create a custom JSON classifier and specify the JSON path as $[*]

    For more details, here is the link to the AWS documentation - https://docs.aws.amazon.com/glue/latest/dg/custom-classifier.html#custom-classifier-json