Search code examples
c#pollyretry-logic

retry using Polly


Do these 2 retry policies indicate the same?

Policy
    .Handle<SomeExceptionType>()
    .WaitAndRetry(
        new[]
        {
            TimeSpan.FromMinutes(1),
            TimeSpan.FromMinutes(1),
            TimeSpan.FromMinutes(1)
        });
Policy
    .Handle<SomeExceptionType>()
    .WaitAndRetry(
        3,
        retryAttempt => TimeSpan.FromMinutes(1)
    );

Solution

  • Yes, they do the same.

    Both code defines a policy which will execute the same operation at most 4 times: the initial attempt + three additional attempts.

    The main difference between the two overloads is the following:

    • The former defines the penalties in a static way
      • it predefines the delays between the different attempts
    • The latter defines the penalties in a dynamic way
      • it can compute the delays based on which retry is about to happen

    In your particular example your second alternative can be defined like this:

    Policy
        .Handle<SomeExceptionType>()
        .WaitAndRetry(
            3,
            _ => TimeSpan.FromMinutes(1)
        );
    

    With the discard operator you are explicitly stating that you are not using that parameter inside the sleepDurationProvider to calculate the new delay.


    For the sake of clarity I've used the penalty, delay and sleep terms interchangeable in this post.


    UPDATE #1

    Here are two examples where you can take advantage of the dynamic penalty calculation.

    Exponential backoff + jitter

    Rather than waiting the same amount of time between each attempts it is advisable to use larger and larger delays to give space for the downstream system to selfheal. So, for example: 2, 4, 8 ...

    The jitter is just a small random number to avoid that all clients are trying to send their retry attempts at the same time. So it scatters/disperse the clients' retry attempts in time.

    const int maxDelayInMilliseconds = 32 * 1000;
    var jitterer = new Random();
    Policy
      .Handle<HttpRequestException>()
      .WaitAndRetryForever(
          retryAttempt =>
          {
              var calculatedDelayInMilliseconds = Math.Pow(2, retryAttempt) * 1000;
              var jitterInMilliseconds = jitterer.Next(0, 1000);
    
              var actualDelay = Math.Min(calculatedDelayInMilliseconds + jitterInMilliseconds, maxDelayInMilliseconds);
              return TimeSpan.FromMilliseconds(actualDelay);
          }
      );
    

    Circuit Breaker aware Retry

    If you are using a circuit breaker to avoid flooding downstream system while it's self-healing you can make your retry aware of this.

    By default all policies are independent and they are unaware of each other. If you issue a retry attempt while the CB is open then you will receive a BrokenCircuitException (so it short-cuts the execution). But you can dynamically calculate the delay based on the CB's state so you can skip these unnecessary retries.

    The Circuit Breaker definition

    Policy<HttpResponseMessage>
        .HandleResult(res => res.StatusCode == HttpStatusCode.InternalServerError)
        .CircuitBreakerAsync(3, TimeSpan.FromSeconds(2),
           onBreak: (dr, ts, ctx) => { ctx[SleepDurationKey] = ts; },
           onReset: (ctx) => { ctx[SleepDurationKey] = null; });
    

    The retry definition

    Policy<HttpResponseMessage>
        .HandleResult(res => res.StatusCode == HttpStatusCode.InternalServerError)
        .Or<BrokenCircuitException>()
        .WaitAndRetryAsync(4,
            sleepDurationProvider: (c, ctx) =>
            {
                if (ctx.ContainsKey(SleepDurationKey))
                    return (TimeSpan)ctx[SleepDurationKey];
                return TimeSpan.FromMilliseconds(200);
            });
    

    This advanced use case is described in detail here.