Search code examples
node.jsasynchronousasync-awaitsetintervalgraceful-shutdown

How should a NodeJs "graceful shutdown" handle setInterval?


I have a node backend taking HTTP requests using express. I am shutting down gracefully like this:

process.on( 'SIGINT', function() {
    console.log("SIGINT signal received.");

    server.close(function(err) {
        if (err) {
            console.error(err)
            process.exit(1)
        }
    
        //Stop reoccurring tasks
    
        //Close database connection
    
        process.exit(0);
    });
    process.exit(0);
});

What I have is working fine, but I am concerned about my "Stop reoccurring tasks" step. Elsewhere in my code, I call a function that looks like this:

export async function launchSectionFinalizer() {
    finalizeSections();
    
    //1 hr * 60 min/hr * 60 s/min * 1,000 ms/s = 3,600,000 ms
    return setInterval(finalizeSections, 3_6000_000);
}

Where finalizeSections is an async function that performs a series of database operations (postgres database).

My question is about the nature and behavior of setInterval. How can I make sure that finalizeSections isn't in the middle of its execution when I receive SIGINT? I'm worried that if my program receives SIGINT and closes the server at the wrong time it could catch finalizeSections in the middle of its operations. If that happens, I could end up with those database operations partially complete (ie if I execute a series of sql commands one after another, insert1, insert2, and insert3, I do not want to execute 1 and 2 without also executing 3).

I have done some googling and read something about how node will wait for all of its processes and events to complete before closing. Would that include waiting for my call to finalizeSections to complete?

Also, I am aware of clearInterval, but I am not sure if that function only stops the timer or if it will also cause node to wait for finalizeSections to complete.


Solution

  • Calling clearInterval will only cancel the timer and not wait for finalizeSections to finish.

    Because your graceful shutdown calls process.exit(0) it will not wait for pending asynchronous tasks to finish and it will exit immediately:

    Calling process.exit() will force the process to exit as quickly as possible even if there are still asynchronous operations pending that have not yet completed fully, including I/O operations to process.stdout and process.stderr

    One way to solve this without using any packages is to save a reference to the promise returned by finalizeSections() and the intervalId returned by setInterval():

    intervalId = setInterval(() => {
      finalizeSectionsPromise = finalizeSections();
    }, 3_6000_000)
    

    Then in the shutdown code.

    clearInterval(intervalId);
    if (finalizeSectionsPromise) {
      await finalizeSectionsPromise;
    }
    ...
    process.exit(0);
    

    If you are able to use other packages I would use a job scheduling library like Agenda or Bull, or even cron jobs:
    https://github.com/OptimalBits/bull
    https://github.com/agenda/agenda

    Also take a look a stoppable or terminus to gracefully shutdown servers without killing requests that are in-flight:
    https://www.npmjs.com/package/stoppable
    https://github.com/godaddy/terminus