Search code examples
masstransit

How to implement redelivery with infinite retry until saga is finalized?


We use Masstransit 8 and saga state machines and we have a business flow with multiple sagas communicating with each other and also other external services. We have configured generic message retry for unhandled exceptions that will retry 5 times with a 30 seconds delay.

For longer errors we are looking to implement a redelivery strategy. our business rules allows our sagas to have a TTL of 12 hours. After this we don't want to proceed and instead finalize and remove the saga. we "close" our sagas using Schedule feature that will finalize the saga if it's running after 12 hours.

How can we configure a delayed redelivery (using UseDelayedRedelivery in Masstransit) that will retry every 5 minute for an infinite retry count as long as the saga is not finalized? What i am looking for is that for as long as the saga is alive (12 hours) we want to redeliver, but when the saga is finalized and removed we want to stop redeliver the message (i guess this also means that it will be discarded from the dlq?). The goal is to have a solid process that will recover our sagas in case of unexpected exceptions (such as bugs or external services not responding).

Update:

For a little bit more context. In this specific scenario it is actually a request that I want to redeliver, but I fail to understand on what level I can configure this. Here goes a simplified example:

public class SurveySaga : MassTransitStateMachine<SurveyState> 
{
    public Event<RequestSurveyCommand> Requested { get; private set; }

    public Request<SurveyState, GetConfig, GetConfigResponse> GetConfigRequest { get; private set; }
    
    public SurveySaga()
    {
         Initially(
            When(Requested)                    
                 .Request(GetConfigRequest, context => new GetSomeConfigRequest { Foo = "bar" })
        );
    }
}

So, if the GetConfigRequest fails we can handle this by a When(GetConfigRequest.Faulted). On what level can we discard this request if the saga is finalized? In my tests the Faulted method is never hit if I add an infinite re-delivery, I guess that is intentional? Ideally I would like to simply stop re-delivering the request if the saga is finalized/removed.

Update 2:

From my understanding, in my scenario, it is not possible to configure it to discard on a missing instance because this is a request and not an event. This is how the request is configured in the saga:

    Request(() => GetConfigRequest, x => x.GetConfigRequestId, cfg =>
    {
        cfg.ServiceAddress = _endpointer.GetAddress(EndpointName.GetConfig);
        cfg.Timeout = TimeSpan.FromSeconds(_options.RequestTimeoutInSeconds);
    });

There is no configurator available to configure what to do on a missing instance. The request is correlated to the saga by the GetConfigRequestId specified in the expression. How would I be able to correlate this request and discard it if the instance is missing? It seems to be pretty straight forward for an event, but not so straight forward for a request.


Solution

  • Configure delayed redelivery such that the retry limit * retry delay >= 12 hours. If the message is redelivered and the saga has already finalized (either in the Final state, or missing from the repository) complete the message instead of throwing an exception, so that the message is not again scheduled for redelivery.

    To simply discard the redelivered message when the saga instance is missing, you can use the OnMissingInstance configuration method.

    Request(() => GetConfigRequest, x => x.GetConfigRequestId, cfg =>
    {
        cfg.ServiceAddress = _endpointer.GetAddress(EndpointName.GetConfig);
        cfg.Timeout = TimeSpan.FromSeconds(_options.RequestTimeoutInSeconds);
    
        cfg.Completed = m => m.OnMissingInstance(i => i.Discard());
    });
    

    Updated to show syntax added in v8.0.14