Akka Circuit Breaker sharing between actors

Question

I have a shared external resource (say a file store) which a pool of actors is using. Each time a new request is made to the file store a new actor is created to fill the request with a reference to the external system passed in.

The current approach where I create a circuit breaker per actor defeats the purpose as a new actor is created for each 'request' which performs a sequence of operations on this external resource.

Not ideal - too many CB instances;

class MySharedResourceActor(externalResourceRef: ExtSystem) extends Actor with ActorLogging {
   val breaker = new CircuitBreaker(context.system.scheduler,
     maxFailures = 5,
     callTimeout = 10.seconds,
     resetTimeout = 1.minute).onOpen(notifyMeOnOpen())

   def receive = {
      case SomeExternalOp =>
          breaker.withSyncCircuitBreaker(dangerousCallToExternalSystem()) pipeTo sender()
   }
} 

Better Approach - pass in a CB ref;

class MySharedResourceActor(externalResourceRef: ExtSystem, val breaker: CircuitBreaker) extends Actor with ActorLogging {
   def receive = {
      case SomeExternalOp =>
          breaker.withSyncCircuitBreaker(dangerousCallToExternalSystem()) pipeTo sender()
   }
} 

Is it safe to pass in a Circuit-breaker reference from the parent actor which also maintains a reference to the external system and share this circuit breaker between multiple actors in a router pool, dynamically created or otherwise?

Answer 1

Yes it's safe to follow this approach. We share circuit breakers across related actors (pooled or otherwise) that are making http calls to the same host. If you didn't do this, and let each instance have it's own breaker, even if they were long lived instances, each one would need to hit the fail threshold separately before the breaker opened and I doubt this is the behavior you want. By sharing, it allows multiple actors to contribute stats (fails, successes) into the breaker so that the breaker is representative of all calls that have gone into the resource.

In looking at Akka's code, they are using atomics inside of the circuit breaker to represent state and handle state transitions, so they should be safe to use in multiple actors.