Search code examples
azureazure-eventhub

Azure EventHub Event Format


All,

I set up an EventHub Namespace and EventHub and was able to successfully send and receive events to it using Python scripts. I was also able to enable the Capture feature and store the events in Azure Blob Storage in Avro format.

I wanted to know more about EventHubs and would be great if someone points me in the right direction:

[1] During Capture, can the file format be changed to say for example csv or json format so that the files get created in csv/json format? Or is Avro the only one supported?

[2] When the Event is in EventHub itself before it gets aged out what is the format of the event - is it json?

[3] When the Event data is stored in Azure Blob Storage (in avro format) how do I know what is the structure of the data so that I can create an appropriate schema which I can then use in Azure Data Explorer definition to load data into Kutso database.

Thanks,

rgn


Solution

  • Regarding your questions;

    1) Event Hubs Capture writes only in AVRO format today.

    2) Service doesn't know about the payload format it is processing and it stores the events in binary format.

    3) AVRO format specifies the schema at the top of the file content if this is what you are asking for. You may check here regarding the format details - https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-capture-overview#exploring-the-captured-files-and-working-with-avro