Akka and blocking operations

Question

I have an akka system where 99% of request involve blocking DB calls. I was wondering if there is any benefit in such system of executing blocking operations in separate dispatcher\threadpool, as described in best practices.

While akka threads will be idle waiting, the threadpool task queue can be either overflow memory or start to reject new tasks, which an actor has ho handle somehow.

I understand that this is common question, but I cannot find any relevant article or discussion.

Answer 1

The Akka project is very supportive of the Reactive Manifesto. The documentation assumes that by using Akka you are also interested in building a system according to the Reactive Manifesto.

A key part of the Reactive Manifesto is that systems should remain responsive at all times. If you used the same dispatcher/thread pool for the blocking DB operations as for your main Akka routing infrastructure, it is possible that all the Akka threads could be occupied by the DB operations, and your system would effectively be deadlocked until one of the DB operations completes. This might not be a problem for a simple system on a single JVM that only performs this task, but it's easy to think of new features that would make this a problem:

1. Suppose you wanted to add an admin page that showed the number of DB requests currently in flight and the number that are waiting to be executed by one of the DB threads. This page should remain responsive at all times, even when all the DB threads are blocked, therefore the code that serves this page should use a different dispatcher that can't be blocked by the DB.
2. Suppose you wanted to add multiple JVMs to your system accross multiple different machines, either to increase scale or to improve reliability with some fault tolerance. This involves various gossip messages and heartbeats to be sent across the network. If these system messages were delayed because of the blocking DB operations then the cluster would not work correctly.

A separate dispatcher could be useful for another reason. You probably want to throttle access to the database so that your system does not overload the database. You could make the DB dispatcher a fixed size thread pool and use that as a crude but effective way to throttle your database requests. The alternative would be to implement some buffers withing your actors but this will require a lot of extra code (although you might have to do this anyway if you are worried about running out of memory because of too many tasks etc).