Deleting a message from Azure Queue Service by value

Question

I'm using an Azure Storage Queue Service to track a queue of long running jobs which are en-queued by multiple disparate clients, but I now have a requirement to remove messages from the queue if any exist matching some given criteria.

I realise that this is somewhat anti-pattern for a queue, but the service does provide some functionality in addition to simple queueing (such as Delete Message and Peek Messages) so I thought I'd try to implement it.

The solution I've come up with works, but is not very elegant and is quite inefficient - and I'm wondering if it can be done better - or if I should bin the whole approach and use a mechanism which supports this requirement by design (which would require a fair amount of work across different systems). Here is the simplified code:

        var queue = MethodThatGetsTheAppropriateQueueReference();
        await queue.FetchAttributesAsync(); //populates the current queue length
        if (queue.ApproximateMessageCount.HasValue)
        {
            // Get all messages and find any messages to be removed.
            //   this makes those messages unavailable for other clients 
            //   for the visibilityTimeOut period.
            //   I've set this to the minimum of 1 second - not ideal though
            var messages = await queue.GetMessagesAsync(queue.ApproximateMessageCount.Value);
            var messagesToDelete = messages.Where(x => x.AsString.Contains(someGuid));

            // Delete applicable messages
            messagesToDelete.ToList().ForEach(x => queue.DeleteMessageAsync(x));                
        }

Note originally I tried using PeekMessagesAsync() to avoid affecting messages which do not need to be deleted, but this does not give you a PopReceipt which is required by DeleteMessageAsync().

The questions:

1. Is there a way to do this without pulling ALL of the messages down? (there could be quite a few)
2. If 1 isnt possible, is there a way to get the PopReceipt for a message if we use PeekMessagesAsync()?

Answer 1

Is there a way to do this without pulling ALL of the messages down? (there could be quite a few)

Unfortunately no. You have to Get messages (a maximum of 32 at a time) and analyze the contents of the messages to determine if the message should be deleted.

If 1 isnt possible, is there a way to get the PopReceipt for a message if we use PeekMessagesAsync()?

Again, No. In order to get PopReceipt, a message must be dequeued which is only possible via GetMessagesAsync(). PeekMessagesAsync() simply returns the message without altering its visibility.

Possible Solution

You may want to look into Service Bus Topics and Subscriptions for this kind of functionality.

What you could do is create a topic where all messages will be sent.

Then you would create 2 subscriptions: In one subscription you will set a rule which checks for message contents for the matching value and in other subscription you will set a rule which checks for message contents for not matching value.

What Azure Service Bus will do is check each message that arrives against the rules and accordingly pushes the message in appropriate subscription. This way you will have a nice separation of messages that should/shouldn't be deleted.