Search code examples
c#.netbatch-processingazureservicebusazure-servicebus-queues

Best practice for receiving large messages on ServiceBus though Batch


I am using ReceiveBatch API to process the messages(Batch Processing) from Azure Service Bus. In this I noticed following.

When message size increased number of messages received in a batch gets decreased.

For examples

  1. When BatchSize =50, MessageSize = 20Kb and Prefetch Count = 50 ===> Number of messages received while Calling serviceBusClient.ReceiveBatch = 12 messages

  2. When BatchSize =50, MessageSize = 10Kb and Prefetch Count = 50 ===> Number of messages received while Calling serviceBusClient.ReceiveBatch = 22 messages

  3. When BatchSize =50, MessageSize = 5Kb and Prefetch Count = 50 ===> Number of messages received while Calling serviceBusClient.ReceiveBatch = 44messages

In all above 3 scenarios, around 220 to 240 KB messages I can receive at a time. So here what are the best practices to receive the more numbers of messages regardless of message size ? in batch processing(serviceBusClient.ReceiveBatch)


Solution

  • The maximum size of the messages that can be processed in batch is 256 KB. The batch gets rejected if the size id greater then 256 KB.

    Irrespective of single or batch processing, you can only send or receive the messages within 256 KB.

    To make the size of the message deterministic, add the real message content to a blob storage and send the blob URL as the message body. Here, the size of all the messages will be same and very less.

    Thus, you can determine the number of messages that can be processes per batch.

    This can also be achieved using Service Bus Plugins.