Search code examples
c#azureservicebus.net-8.0

Processing of a Azure Service Bus Queue, to empty it in one go


I have a Azure Service Bus queue, which possibly wrongly I am using to store ID's for views of a particular item. Each time a view is received, I send a message to the queue for processing later.

When later comes, I want to ideally process ALL messages in the queue so I can batch the views update to the Database. There seem to be a couple of options, either to spin up a ServiceBusProcessor, subscribe to the ProcessMessageAsync, call await processor.StartProcessingAsync(); and let it run for an amount of time, before calling await processor.StopProcessingAsync(); whilst in the method dealing with the message processing batches up the changes to push to the Db.

The other option is using a ServiceBusReceiver and calling await receiver.ReceiveMessageAsync(); in a loop until the result of that is null (denoting the queue is empty, as far as I can tell), whilst batching up each received message to update the Db.

My question is, is either of these preferred over the other?

I know that using the processor, I effectively have to guess how long to process the queue in the hope it will be long enough to empty, which isn't entirely ideal, but does let me set the prefetch count, receive mode (ReceiveAndDelete I went for) and ConcurrentCalls. Which seems to help the overall processing speed.

Using a loop with the receiver.ReceiveMessageAsync appears to meet my use case better, but I am unsure of how reliable this is to exhaust the whole queue in one go.

any help/pointers much appreciated.


Solution

  • Purging a queue is a requested feature you can upvote. Unfortunately, until it's available, you'll need to drain the queue yourself. How you do it is up to you but I would suggest the following.

    1. Use receiver.ReceiveMessagesAsync() to retrieve more than one message
    2. Use ReceiveAndDelete mode
    3. Maximize parallelism