Search code examples
pythonazurecommitoffsetazure-eventhub

Is it possible to continue reading from where receiver stopped in EventHub with Python?


I'm using the official azure-eventhub lib for reading events from a topic from EventHub. I want to be able when the receiver stop reading, EventHub to keep the last offset of my receiver group, so when I will start reading again I can start from where I stopped. That is in Kafka done with the commit policy, but I can't find anything similar in the azure-eventhub lib. Is there anything like this in the azure-eventhub or any other python lib for eventhub?


Solution

  • You can use Event Processor Host in python to set the checkpoint to blob storage.

    The detailed sample code is here in github, we're using it for checkpoint purpose.

    Before using Event Processor Host, you should have/create an azure storage account which is used to save the checkpoint. And create a container inside that azure storage account. The storage account / account key / container are used here in the sample code.

    In the sample, this line of code context.checkpoint_async() is used to set checkpoint.

    Please let me know if you have more questions about this.