Search code examples
azureservicebus

How can I speed up posting to ServiceBus?


The following Azure Function is running on Azure (UK South) and posting messages to a Service Bus that is also in UK South.

This is taking 25 seconds, which means I am getting a rate of approximately 40 messages per second, which is nowhere near enough.

Please note that the actual requirement I have is to send messages sequentially to various queues/topics in order they are found in a database, so I cannot use batch send.

Is there a way to speed this up, as 40 per second is not fast enough?

using System.Diagnostics;
using System.Net;
using Azure.Messaging.ServiceBus;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Azure.Functions.Worker.Http;
using Microsoft.Extensions.Logging;
 
namespace Speedtest
{
    public class Function1
    {
        private readonly ILogger _logger;
 
        public Function1(ILoggerFactory loggerFactory)
        {
            _logger = loggerFactory.CreateLogger<Function1>();
        }
 
        [Function("Function1")]
        public async Task<HttpResponseData> Run([HttpTrigger(AuthorizationLevel.Function, "get", "post")] HttpRequestData req)
        {
            var client = new ServiceBusClient("...omitted...");
 
            var sender = client.CreateSender("customercreated");
 
            var sw = Stopwatch.StartNew();
 
            for (int i = 0; i < 1000; i++)
            {
                var message = new ServiceBusMessage($"Hahaahahaha{i}");
                await sender.SendMessageAsync(message);
            }
 
            sw.Stop();
 
            var response = req.CreateResponse(HttpStatusCode.OK);
            response.Headers.Add("Content-Type", "text/plain; charset=utf-8");
 
            response.WriteString($"Time taken was {sw.ElapsedMilliseconds}");
 
            return response;
        }
    }
}

Solution

  • The solution I came up with was to have a topic named events-ingestion and post to that in bulk.

    Then I have multiple subscriptions on that topic with filter criteria like Target = 'customer-created' with a rule to forward the messages to the customer-created topic.

    This way I am seeing a massive throughput even from my remote machine.