Search code examples
amazon-web-servicesaws-lambdatimeout

can we use recursion with AWS lambda function that it stops execution within 15 minutes time limit and save the point at which it stopped?


  1. track the time elapsed in lambda function stop it at 9-10 min .
  2. save the point at which it stopped and continue untill task is completed
  3. compulsory use lambda function

Solution

  • I support John Rotenstein's response, stating that if you're using lambda this way, you probably shouldn't be using lambda. Out of my own curiosity though, I think the code you'd be looking for is something along the following lines (written in Node.JS)

    let AWS = require('aws-sdk');
    let functionName = 'YOUR_LAMBDA_FUNCTION_NAME'
    let timeThreshholdInMillis = 5000 // 5 seconds
    
    exports.handler = async (event, context, callback) => {
        let input = JSON.parse(event['foo']); // input from previous lambda, omitted try-catch for brevity
        let done // set this variable to true when your code is done
    
        let interval = setInterval(() => {
            if (done) callback(null, true)
    
            let remainingTime = context.get_remaining_time_in_millis();
    
            if (remainingTime < timeThreshholdInMillis) {
                // Going to restart
                clearInterval(interval);
                // save your progress (local or remote) to `input` variable
                let lambda = new AWS.Lambda();
                return lambda.invoke({
                    FunctionName: functionName,
                    Payload: JSON.stringify({ 'foo': input }) // Pass your progress here
                }, (err, data) => {
                    // kill container
                    if (err) callback(err)
                    else callback(null, data)
                });
            }
        }, 1000);
    }
    

    Edit: an example

    This is to clarify how 'passing progress' would work in recursive lambdas.

    Let's say you want to increment a variable (+1) every second and you want to do this in Lambda.

    Givens We will increment the variable by 1 once every 1000 ms. Lambda will run until remaining time is < 5000 ms. Lambda execution timeout is 60,000 ms (1 minute).

    Lambda function pseudo code:

    function async (event, context) {
        let counter = event.counter || 0
    
        setInterval(() => {
            if (context.get_remaining_time_in_millis() < 5000) {
                 // start new lambda and pass `counter` variable
                 // so it continues counting and we don't lose progress
                 lambda.invoke({ payload: { counter } })
            } else {
                // Add 1 to the counter variable
                counter++
            }
        }, 1000)
    
    }