Search code examples
azure-functionscold-start

Are Azure Functions cold starts for each client?

Azure Functions that run on consumption plan have a cold start when the function has not been run for a while, I think its after 20 minutes. Its an extra latency for the client for about 10-20 seconds.

I wonder if this cold start is per each client? I mean if a function has run by client 1, and then another client 2 runs it again, does also client 2 gets a cold start?

If not, I guess that if I have clients continuously using my function all day long, then there will be no cold starts at all (except just for the first client of course)? And then cold starts will not be a problem for applications that are heavily used.

Solution

  • The cold start is per Function App instance, not per user. A cold start occurs when an application is shut down due to no usage. Like we can also see in ASP.NET web applications running on IIS.

    For information on Cold start behavior of the different hosting plans, see Azure Functions hosting options - Cold start behavior.

    For Consumption plan, the cold start behavior is as follows:

    Apps may scale to zero when idle, meaning some requests may have additional latency at startup. The consumption plan does have some optimizations to help decrease cold start time, including pulling from pre-warmed placeholder functions that already have the function host and language processes running.

    The below is taken from the article Understanding Serverless cold start. The article is a bit old, but still relevant.

    What is cold start?

    Broadly speaking, cold start is a term used to describe the phenomenon that applications which haven’t been used take longer to start up. In the context of Azure Functions, latency is the total time a user must wait for their function. From when an event happens to start up a function until that function completes responding to the event. So more precisely, a cold start is an increase in latency for Functions which haven’t been called recently. When using Azure Functions in the dedicated plan, the Functions host is always running, which means that cold start isn’t really an issue. So, our scope is narrowed to Functions running the serverless consumption model. Let’s go deeper.

    What happens during cold start

    What happens during cold start

    1. Azure allocates a preconfigured server from the pool of warm workers to your app. This server already has the Functions runtime running on it, but it is unspecialized.

    2. This worker becomes specialized by configuring the Functions runtime in ways that are specific to your app. A few things happen to do this specialization, including:

      • The Azure Functions infrastructure mounts your Azure Files content to the worker you’ve been assigned
      • App settings specific to your function app are applied to the worker

    3. The Functions runtime resets, and any required extensions are loaded onto the worker. To figure out which extensions to load, the runtime reads the function.json files of any function in the function app. For instance, this happens if you’re using Durable Functions, or if you have input or output bindings.

    4. The Functions themselves are loaded into memory by language providers. This will take a varying amount of time based on the size of your application.

    5. Your code runs.

    EDIT:

    1. If I have multiple Functions in my app, does each Function has its own cold start or is it one cold start for the entire app?
      Cold start is on Function App level, not on individual Function level.
    2. According to this I guess that my assumption is correct, that cold starts will then be no problem for heavily used apps, right?
      That is correct.
    3. Is it guaranteed that two clients that run a function uses the same instance of the function app? Or can there be different instances for some reasons and therefore cause a cold start for both clients?
      Azure Functions scale based on load. If the load increases, you get extra instances. And those extra instances have their own (cold) start, of course.
      So: if the load fits on one instance, all executions are probably on the same one. If multiple instances are there, there's no saying where the request will end up.

    EDIT 2:
    If I have for example an app with 100 Functions, will the cold start be longer than for an app with 10 Functions
    It depends. In general, my answer would be yes because probably you will have more dependencies, more external resources to connect to and overall more code to load. But if you keep the dependency tree small, there shouldn't be too much difference.

    Taken from the Write lightweight code chapter of the article I referred to earlier:

    Dependencies: When you deploy your code, your dependencies are added as files to your application. [...] all code needed by your app eventually gets loaded into memory, which takes longer with a larger application. So, if you have a ton of dependencies, you’ll get a longer cold start due to increased time for I/O operations from Azure Files, and longer time needed to load a bigger app into memory. [...]

    Efficient Code: Sometimes the answer is simply writing more efficient code. There are a few approaches to note here, first try to make as much processing as possible asynchronous. Functions won’t perform well if you have a heavyweight synchronous call blocking your code from completing. Along this vein, try to minimize the amount of work that has to happen before your code starts up and avoid code which consumes a lot of CPU. [...]