Search code examples
node.jsexpresspollycircuit-breakerretrypolicy

How to implement circuit breaker and retry policy in an express app


I have an express backend and i want to implement a circuit breaker and retry policy for each endpoint. I have explored a few libraries like polly.js and opossum but as far as i understand polly.js only allows implementing retry policies and opossum only allows implementing circuit breaking. Please correct me if im wrong with above statements since their documentation isn’t very good.

For anyone answering please provide a code snippet for any dummy endpoint.


Solution

  • As @PeterCsala mentioned, we can use cockatiel in an express app to implement retry and circuit breaker policies.

    For this purpose I made a policy wrapper file as below:

    import {
        ConsecutiveBreaker,
        ExponentialBackoff,
        retry,
        handleAll,
        circuitBreaker,
        wrap,
    } from 'cockatiel';
    
    // Create a retry policy that'll try whatever function we execute 3
    // times with a randomized exponential backoff.
    const retryPolicy = retry(handleAll, {
        maxAttempts: 3,
        backoff: new ExponentialBackoff(),
    });
    
    // Create a circuit breaker that'll stop calling the executed function for 10
    // seconds if it fails 5 times in a row. This can give time for e.g. a database
    // to recover without getting tons of traffic.
    const circuitBreakerPolicy = circuitBreaker(handleAll, {
        halfOpenAfter: 10 * 1000,
        breaker: new ConsecutiveBreaker(5),
    });
    
    const combinedPolicy = wrap(retryPolicy, circuitBreakerPolicy);
    
    retryPolicy.onRetry((context) => {
        console.log(`Retry on atempt #${context.attempt} after ${context.delay}ms`);
    });
    
    circuitBreakerPolicy.onBreak(() => {
        console.log('Circuit breaker open');
    });
    circuitBreakerPolicy.onHalfOpen(() => {
        console.log('Circuit breaker half open');
    });
    
    // combinedPolicy.onSuccess((error) => {
    //     console.log('acceptable under the policy', error);
    // });
    combinedPolicy.onFailure((error) => {
        console.log('policy failed');
        console.log(error);
    });
    
    export default combinedPolicy;
    

    Now, we can wrap any function in this wrapper to implement these policies on it like below:

    await combinedPolicy.execute(() => functionWhichMightThrowError())