My async consumer fetching the data from the Event Hub. I am updating the checkpoint after every event processing. When any error/exception is raised while processing the event then not updating the checkpoint. So that we can get the same event later and we'll not skip those events. But, we are not getting these processing failed events until we did a server restart.
Once the server restarts we are getting the new events along with events that are not checkpointed.
Is there any way we can get them (Events that are not checkpointed) without restart?
And also I am confused a little with checkpoint behavior. For suppose
Event 1 is processed. Checkpoint.
Event 2 throws an exception.
Event 3 throws an exception.
Event 4 is processed. Checkpoint.
Now whenever the server restarts, the checkpoint is past Event 2 and 3, and therefore they will never be reprocessed. But, how come they are coming back again when the server restart as the checkpoint is updated at Event 4, it should fetch from Event 4 right?
The short answer is "not from the same consumer."
Event Hubs is an event stream that can be read in a forward-only manner; you cannot rewind the stream and re-read events in-place. To rewind, you have to create a new link and specify an earlier starting position in the stream.
The consumer clients from the Event Hubs SDK read the events and emit them to your code for processing. Each event read is emitted once - its the responsibility of the processing code to judge how to do so. If your processing fails in a manner that you deem transient, ensuring retries for that event should be done as part of the processing logic, as would any form of dead-lettering for events that you would like to revisit later.