Order Process with Azure Durable Functions or not

Question

I am creating an architecture to process our orders from an ecommerce website who gets 10,000 orders or more every hour. We are using an external third party order fulfillment service and they have about 5 Steps/APIs that we have to run which are dependent upon each other.

I was thinking of using Fan in/Fan Out approach where we can use durable functions.

My plan

1. Once the order is created on our end, we store in a table with a flag of Order completed.
2. Run a time trigger azure function that runs the durable function orchestrator which calls the activity functions for each step

Now if it fails, timer will pick up the order again until it is completed. But my question is should we put this order in service bus and pick it up from there instead of time trigger.

Because there can be more than 10,000 records each hour so we have to run a query in the time trigger function and find orders that are not completed and run the durable orchestrator 10,000 times in a loop. My first question - Can I run the durable function parallelly for 10,000 records?

If I use service bus trigger to trigger durable orchestrator, it will automatically run azure function and durable 10,000 times parallelly right? But in this instance, I will have to build a dead letter queue function/process so if it fails, we are able to move it to active topic

Questions:

1. Is durable function correct approach or is there a better and easier approach?
2. If yes, Is time trigger better or Service bus trigger to start the orchestrator function?
3. Can I run the durable function orchestrator parallelly through time trigger azure function. I am not talking about calling activity functions because those cannot be run parallelly because we need output of one to be input of the next

Answer 1

This usecase fits function chaining. This can be done by

• Have the ordering system put a message on a queue (storage or servicebus)
• Create an azure function with storage queue trigger or service bus trigger. This would also be the client function that triggers the orchestration function
• Create an orchestration function that invokes the 5 step APIs, one activity function for each (similar to as given in function chaining example.
• Create five activity function, one f for each API

Ordering system

var clientOptions = new ServiceBusClientOptions
{ 
    TransportType = ServiceBusTransportType.AmqpWebSockets
};

//TODO: Replace the "<NAMESPACE-NAME>" and "<QUEUE-NAME>" placeholders.
client = new ServiceBusClient(
    "<NAMESPACE-NAME>.servicebus.windows.net",
    new DefaultAzureCredential(),
    clientOptions);

sender = client.CreateSender("<QUEUE-NAME>");

var message = new ServiceBusMessage($"{orderId}");

await sender.SendMessageAsync(message);

Client function

public static class OrderFulfilment
{
    [Function("OrderFulfilment")]
    public static string Run([ServiceBusTrigger("<QUEUE-NAME>", Connection = "ServiceBusConnection")] string orderId,
        [DurableClient] IDurableOrchestrationClient starter)
    {
        var logger = context.GetLogger("OrderFulfilment");

        logger.LogInformation(orderId);

        return starter.StartNewAsync("ChainedApiCalls", orderId);
    }
}

Orchestration function

[FunctionName("ChainedApiCalls")]
public static async Task<object> Run([OrchestrationTrigger] IDurableOrchestrationContext fulfillmentContext)
{
    try
    {
        // .... get order with orderId
        var a = await context.CallActivityAsync<object>("ApiCaller1", null);
        var b = await context.CallActivityAsync<object>("ApiCaller2", a);
        var c = await context.CallActivityAsync<object>("ApiCaller3", b);
        var d = await context.CallActivityAsync<object>("ApiCaller4", c);
        return  await context.CallActivityAsync<object>("ApiCaller5", d);
    }
    catch (Exception)
    {
        // Error handling or compensation goes here.
    }
}

Activity functions

[FunctionName("ApiCaller1")]
public static string ApiCaller1([ActivityTrigger] IDurableActivityContext fulfillmentApiContext)
{
    string input = fulfillmentApiContext.GetInput<string>();

    return $"API1 result";
}

[FunctionName("ApiCaller2")]
public static string ApiCaller2([ActivityTrigger] IDurableActivityContext fulfillmentApiContext)
{
    string input = fulfillmentApiContext.GetInput<string>();

    return $"API2 result";
}

// Repeat 3 more times...