How to mock ServiceBusClient with retry options

Question

I'm trying to ensure that the retry functionality is working correctly for ServiceBusSender.SendMessageAsync(). After a number of retries I need to do something else.

Currently I'm doing the following to mock both the ServiceBusClient and ServiceBusSender but when I step through the code I can't see SendMessageAsync() being called the number of retries I would expect.

var serviceBusClientMock = new Mock<ServiceBusClient>(
    It.IsAny<string>(), new ServiceBusClientOptions()
    {
        RetryOptions = new ServiceBusRetryOptions()
        {
            Mode = ServiceBusRetryMode.Fixed,
            Delay = TimeSpan.FromSeconds(3),
            MaxDelay = TimeSpan.FromSeconds(10),
            MaxRetries = 3
        }
    }
);

var serviceBusSenderMock = new Mock<ServiceBusSender>();
serviceBusSenderMock
    .Setup(x => x.SendMessageAsync(It.IsAny<ServiceBusMessage>(), It.IsAny<CancellationToken>()))
    .Throws(new ServiceBusException("Region Down", ServiceBusFailureReason.ServiceTimeout));   

serviceBusClientMock
    .Setup(x => x.CreateSender(It.IsAny<string>()))
    .Returns(serviceBusSenderMock.Object);

Answer 1

Much of this has been teased out via comments, but to offer confirmation:

This sounds like the retires have already occurred within the SendMessageAsync() method so I won't be able to see them occurring in my code.

This is correct.

Client retries are implicit; your application has no idea that they're taking place other than the operation takes longer to complete. In this case, when control returns to your application after the "SendMessageAsync" call, all retries have already been applied. If your application sees an exception, either that exception was terminal (and not retried) or all of the retry attempts were exhausted before the operation was successful.

The easiest way for your application to observe exceptions that are not otherwise surfaced is through the SDK logs. Context for enabling them can be found in this sample.

You could also create a custom retry policy that you'd pass via the ServiceBusClientOptions used to create your client. The RetryOptions member there has a CustomRetryPolicy property that allows you to specify your own implementation of the abstract ServiceBusRetryPolicy class. This would allow your application to observe all exceptions and own the decision for whether or not to retry. That said, unless you copied logic from our built-in BasicRetryPolicy, you would potentially be altering standard behavior.

not sure what would happen if you provide a wrong namespace

As mentioned in the comments, the best place to understand error scenarios is by taking a look at the Exception Handling section of the Overview docs.

If you've got the wrong namespace configured or the client cannot reach the namespace, you'll either see an exception from the .NET networking stack, a TimeoutException or an OperationCanceledException depending on where in the network/transport stack the exception originates from.

In the case of an incorrect a topic, you would see a ServiceBusException with its Reason property set to ServiceBusFailureReason.MessagingEntityNotFound. This is not retriable.

Simulate a topic being unavailable

This isn't a documented scenario by the service, as entities such as topics are monitored for health and will automatically recover. If a topic is unavailable for more than a couple of seconds, it is very likely that there is something very wrong and impacting multiple data centers.

During any migration or recovery of a topic node, the service call will experience a transient failure (either timeout, service busy, or a general communication failure) - which will be handled by the client retry policy.

One caveat to mention here is that it is possible to manually disable a topic via the Azure Portal. In this case, the failure reason for the ServiceBusException is set to MessagingEntityDisabled. This is not retried and it requires manual action to resolve.