Search code examples
pythonjsontype-conversionavroazure-eventhub

Convert an avro file that was originally a json back to json with Python


So I'm trying to read a JSON file that I send via HTTP POST to an Event Hub that captures the data and converts it to an Avro file format in an Azure Blob Storage, now I'd like to download that file again and convert it back to a JSON format using Python. Is using a JSON serializer the only way to do this?

Initial JSON format


{
  "id": 1,
  "number": "111111111",
  "payload": {
    "intial": "This is a example",
    "BoxId": 2,
    "PersonId": 2,
    "GUID": "1s3q1d-s546dq1-8e22e",
    "borderId": 2,
    "ServiceId": 2,
    "Lat": -63.546547,
    "Lon": -63.546547,
    "TimeStamp": "2021-03-18T08:29:36.758Z",
    "Recording": "azezaerazre",
    "Env": "TEST"
  },
  "operator": 123456,
  "sender": "MSD",
  "binary": 1,
  "sent": "2021-03-29T08:29:36.758Z"
}

Solution

  • Once you have downloaded the Avro file, you should be able to output the records as Avro JSON by using fastavro and doing something like this:

    from fastavro import reader, json_writer
    
    with open("json_file.json", "w") as json_file:
        with open("avro_file.avro", "rb") as avro_file:
            avro_reader = reader(avro_file)
            json_writer(json_file, avro_reader.writer_schema, avro_reader)