Aws lambda running out of memory

Question

I am running an aws lambda (dotnet6 runtime) that gathers information from several databases, serializes it, and exports it to an S3 bucket.

Despite changing memory configuration (up to 10240 MB), cloudwatch logs still report Out of Memory. The code in question contains a method that runs a loop of sql connections in using blocks, and I have realized that this error shows up after 1500 iterations approximately. Even if I skip code ran before this method, the error still happens at the same iteration.

Here is the relevant code:

public async Task FunctionHandler(ILambdaContext context)
{
    context.Logger.LogInformation($"Date: {DateTime.UtcNow:dd/MM/yyyy HH:mm}");
    try
    {            
        List<Empresa> empresas = _servicioEmpresa.ObtenerInformacionBasica();
        _servicioEmpresa.ObtenerInformacionEmpresas(empresas, context);                        

        await GuardarJSonEnS3(JsonSerializer.Serialize(empresas), "empresas", context);            
    catch (Exception ex)
    {
        context.Logger.LogError($"FunctionHandler - {ex}");
    }       
}

public List<Empresa> ObtenerInformacionBasica()
    {
        List<Empresa> empresas = new();
        using (SqlConnection connection = new SqlConnection(_cadenaConexionCentral))
        {
            using (SqlCommand command = new SqlCommand("aws_Analytics_Obtener_Empresas", connection))
            {
                command.CommandType = CommandType.StoredProcedure;
                connection.Open();
                using (SqlDataReader reader = command.ExecuteReader())
                {
                    while (reader.Read())
                    {
                        //fill object
                    }
                }
            }
        }
        return empresas;
    }

public void ObtenerInformacionEmpresas(List<Empresa> empresas, ILambdaContext context)
    {
        foreach (Empresa empresa in empresas)
        {
            context.Logger.LogInformation($"FunctionHandler::ObtenerInformacionEmpresas::Empresa - {empresa.Nombre}");
            try
            {
                using (SqlConnection connection = new SqlConnection(empresa.CadenaConexion))
                {
                    using (SqlCommand command = new SqlCommand("aws_Analytics_Obtener_Informacion_Empresa", connection))
                    {
                        command.CommandType = CommandType.StoredProcedure;
                        connection.Open();
                        using (SqlDataReader reader = command.ExecuteReader())
                        {
                            while (reader.Read())
                            {
                                //fill object
                            }
                        }
                    }
                }                   
            }               
            catch (Exception ex)
            {                   
                context.Logger.LogError($"FunctionHandler::ObtenerInformacionEmpresas - {ex}");
                throw;
            }
        }                        
    }

By knowing the list that comes as parameter, with that logger at line 5 I can see that the error happens at iteration number 1500 aprox (out of 1700 items).

Here are some logs from cloudwatch that illustrate the issue:

As the error suggest, it seems like a memory issue but the max memory usage does not surpass the lambda memory size. As stated before, I have tried max memory configuration and timeout (15 minutes).

Moreover, the lambda runs like a charm on local machine (Windows 10) through mock lambda test tool.

Any help would be much appreciated.

Answer 1

I finally solved the issue. It was not related to aws lambda itself but because of sql server connection pooling.

Here is the issue. As shown in the code within the question description, I had several SQL connections, which translates to more memory usage as pooling means that the connections are held in memory with the expectation of being reused.

By setting pooling to false, every connection gets disposed as soon as it is closed, relieving memory usage. In my case, pooling made no sense as my connection strings were all different.

Hope it makes sense for everyone.

Thanks