Search code examples
c#azureconcurrencyazure-functionsazure-servicebus-queues

Processing Service Bus messages in order (without concurrent calls) in an Azure Function


I need to read and process messages from an Azure Service Bus Queue by an "Azure function". The messages should be handled in the right order so I need to avoid concurrent calls.

I use an Azure Function service bus trigger for this (it's the only subscriber to the queue). According to the documentation I configured the "servicebus/maxConcurrentCalls" (in host.json) setting to 1. On top of this I decorated the function with the "Singleton" attribute. Besides all this the messages seem to be treated in a random order by different threads. What do I miss here? Or did I misunderstand something?

Documentation I used: https://github.com/Azure/azure-webjobs-sdk/wiki/Singleton

host.json:

{
  "serviceBus": {
    "maxConcurrentCalls": 1
  }
}

Azure Function:

using System;
using System.Threading.Tasks;
using Microsoft.ServiceBus.Messaging;

[Singleton]
public static void Run(BrokeredMessage myQueueItem, TraceWriter log)
{
    Stream stream = myQueueItem.GetBody<Stream>();
    StreamReader reader = new StreamReader(stream);
    string messageContentStr = reader.ReadToEnd();

    log.Info($"New TEST message: {messageContentStr} on thread {System.Threading.Thread.CurrentThread.ManagedThreadId}");   

    System.Threading.Thread.Sleep(2000);     
}

Here is an excerpt of the logging. As you can see there are different threads. And, for example, "Message 19" comes before "Message 10". And yes, I'm sure I put the messages in the right order in the queue.

....
2018-05-09T09:09:33.686 [Info] New TEST message: Message 19 on thread 33
2018-05-09T09:09:35.702 [Info] Function completed (Success, Id=007eccd0-b5db-466a-91c1-4f53ec5a7b3a, Duration=2013ms)
2018-05-09T09:09:36.390 [Info] Function started (Id=b7160487-d10d-47a6-bab3-78da68a93498)
2018-05-09T09:09:36.420 [Info] New TEST message: Message 10 on thread 39
...

Solution

  • Take a look and make sure that your Service Bus Queue is NOT partitioned. If it is partitioned, you have multiple message brokers servicing requests, and the order of messaging is not guaranteed. You can read more about it here: https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning#not-using-a-partition-key

    Specifically:

    In the absence of a partition key, Service Bus distributes messages in a round-robin fashion to all the fragments of the partitioned queue or topic. If the chosen fragment is not available, Service Bus assigns the message to a different fragment. This way, the send operation succeeds despite the temporary unavailability of a messaging store. However, you will not achieve the guaranteed ordering that a partition key provides.