How to run long running tasks in azure functions with consumption plan

Question

I have a situation where certain task is running for more than 10 minutes and I have tried breaking that task to chunks of activities. But the orchestrator cannot remain active for more than that of 10 minutes. Can we change the timeout beyond 10 minutes while running the application in consumption plan to begin with

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.DurableTask;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;

public static class OrchestrationFunction
{
    [FunctionName("OrchestrationFunction_TimerTrigger")]
    public static async Task RunOrchestrator(
        [TimerTrigger("0 */5 * * * *",RunOnStartup =true)] TimerInfo timer,
        [OrchestrationClient] IDurableOrchestrationClient starter,
        ILogger log)
    {
        var instanceId = await starter.StartNewAsync("OrchestrationFunction", null);

        log.LogWarning($"Started orchestration with ID = '{instanceId}'");
    }

    [FunctionName("OrchestrationFunction")]
    public static async Task<List<string>> RunOrchestratorFunction(
        [OrchestrationTrigger] IDurableOrchestrationContext context)
    {
        var outputs = new List<string>();
//These activities add up to 18 minutes in total
        outputs.Add(await context.CallActivityAsync<string>("OrchestrationFunction_Activity", null));
        outputs.Add(await context.CallActivityAsync<string>("OrchestrationFunction_Activity", outputs[0]));
        outputs.Add(await context.CallActivityAsync<string>("OrchestrationFunction_Activity", outputs[1]));

        // returns ["guid1", "guid2", "guid3"]
        return outputs;
    }

    [FunctionName("OrchestrationFunction_Activity")]
    public static async Task<string> RunActivity(
        [ActivityTrigger] string input,
        ILogger log)
    {
        // Simulate some delay, e.g., 10 minutes
        await Task.Delay(TimeSpan.FromMinutes(10));

        var newGuid = Guid.NewGuid().ToString();
        log.LogWarning($"Generated GUID: {newGuid}");

        return $"{newGuid},{input}";
    }
}

How can I make this approach feasible with my current consumption plan?

Answer 1

As long as the individual activities don't take longer than 10 minutes, you are good.

Orchestrators do not remain active while the activities are running, they stop running once they reach a point where they need to wait for results/events.

This is what happens in your sample orchestrator at a high level:

1. Orchestrator started
2. Triggers first activity run and writes table row that activity was scheduled -> orchestrator stops
3. First activity runs and produces a result -> sent through queue back to orchestrator
4. Orchestrator starts again, writes activity result to table row and re-executes the orchestrator code; this time it reads from history table that the activity was scheduled and we have a result -> result for Task set so the line that started the first activity is passed
5. Triggers second activity...

So you see it will not matter how long the orchestration itself takes as the orchestrator ever comes up to read/write data to history table and send queue messages to trigger activities.