ConcurrentQueue dequeue problem in Azure function

Question

I have declared a ConcurrentQueue and added a list of GUIDs. Adding into the queue is fine, but when I access the queue from inside the TimerTrigger function it seems like it's empty (updateQueue.count is 0). This behavior is happening in the cloud, but when I execute the same locally it works fine.

public static class Function1
{
    private static readonly ConcurrentQueue<IEnumerable<Guid>> updateQueue = new ConcurrentQueue<IEnumerable<Guid>>();

    public static async Task<IActionResult> UpdateRun(
        [HttpTrigger(AuthorizationLevel.Admin, "post", Route = null)] HttpRequest req, ExecutionContext execContext)
    {
        logger.LogInformation($"FunctionUpdate Start");
        using var reader = new StreamReader(req.Body);
        var request = JsonConvert.DeserializeObject<IEnumerable<Request>>(reader.ReadToEnd());
        var correlationIds = request?.Select(s => s.CorrelationId);

        updateQueue.Enqueue(correlationIds);
        return new OkObjectResult(new Response { HttpStatusCode = HttpStatusCode.Accepted });
    }

    [FunctionName("FunctionHandleQueue"), Timeout("00:05:00")]
    public static async Task HandleQueue([TimerTrigger("0 */1 * * * *")] TimerInfo myTimer, ExecutionContext execContext) // once every 1 minutes
    {
        logger.LogInformation($"before updateQueue condition : {updateQueue.Count}"); 
        if (updateQueue.Count > 0)
        {
            logger.LogInformation($"after updateQueue condition {updateQueue.Count}");

            var guids = new List<Guid>();
            var count = 0;
            while (count <= 1000 && updateQueue.Count > 0)
            {
                updateQueue.TryDequeue(out var updateRequest);
                var enumerable = updateRequest.ToList();
                count += enumerable.Count;
                guids.AddRange(enumerable);
            }

            await new ProcessUpdateSales(CreateMapper(), execContext)
                .Orchestrate(guids)
        }
    }
}

Logs are created when TimerTrigger executes every 1 minute:

before updateQueue condition : 0

Why is updateQueue.Count always 0? What am I doing wrong?

Answer 1

While Azure Functions may typically share a single backplane, it is not guaranteed. Resources can be spun down or up at any time and new copies of functions may not have access to the original state. As a result, if you use static fields to share data across function executions, it should be able to reload the data from an external source.

That said, this is also not necessarily preferable due to how Azure Functions are designed to be used. Azure Functions enable high-throughput via dynamic scalability. As more resources are needed to process the current workload, they can be provisioned automatically to keep throughput high.

As a result, doing too much work in a single function execution can actually interfere with overall system throughput, since there is no way for the function backplane to provision additional workers to handle the load.

If you need to preserve state, use a form of permanent storage. This could take the form of a Azure Durable Function, an Azure Storage Queue, an Azure Service Bus queue, or even a database. In addition, in order to best take advantage of your function's scalability, try to reduce the workload to manageable batches that allow for large amounts of parallel processing. While you may need to frontload your work in a single operation, you want the subsequent processing to be more granular where possible.