Managing Connections in an Azure Serverless Function App

Question

Microsoft recommends you maintain a single instance of CosmosClient across your whole application, and I'm trying to achieve this in my Function App (with more than just CosmosClient). However, even when re-using both database & container proxies, I always see a warning that I have hit the maximum (10) number of connections to Cosmos and that it's discarding the connection when I send through enough requests.

For context, it's a serverless Python Function App triggered by a message queue, the connections are managed in shared code in a helper function. I have to use the Cosmos SDK because I have to both read and update Cosmos doc.

Has anyone successfully navigated this in the past? would it simply be best practice to instantiate a new connection for every single function call? I tried creating a new CosmosClients when receiving burst traffic, but proved very difficult to do efficiently.

Here's an example of the class I'm using to manage connections:

COSMOS_CLIENT = None

class Client:
    def __init__(self):
        self.cosmos_client: CosmosClient = self._get_global_cosmos_client()
    
    def _get_global_cosmos_client(self) -> CosmosClient:
        global COSMOS_CLIENT
        if COSMOS_CLIENT is None:
            logging.info('[COSMOS] NEW CLIENT CONNECTION')
            COSMOS_CLIENT = CosmosClient.from_connection_string(COSMOS_DB_CONNECTION_STRING
        return COSMOS_CLIENT

Answer 1

Conceptually, because you are creating the client based on ConnectionString (there is always 1) this code should always create 1 client.

The number connections is not the number of clients.

Do not create multiple clients, always create 1 client for each account you are interacting against. That single client can perform operations on all existing databases/containers in the account.

Creating multiple clients just creates a problem, because each client will maintain its own independent connections and not reuse them and it will create a higher number of connections than reusing the single client, eventually leading to SNAT port exhaustion.

The error message: Connection pool is full, discarding connection: is not generated by the Cosmos Client directly, rather from the underlying urllib3.connectionpool. See: https://github.com/Azure/azure-sdk-for-python/issues/12102

The CosmosClient supports passing the session through transport, https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/CLIENT_LIBRARY_DEVELOPER.md#transport, -> https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cosmos/azure-cosmos/azure/cosmos/_cosmos_client_connection.py#L198. Reference: https://github.com/Azure/azure-sdk-for-python/issues/12102#issuecomment-645641481