Automatic Retry on unavailable hosts (NoHostAvailableException) in Phantom DSL and Play! 2

Question

I'm currently working on integrating Phantom DSL into a small Play application. Since we're planning on running the app in a Docker environment I'm using Docker Compose on my local machine to test the app.

However, when booting up a Cassandra instance and the Play app at the same time it is unable to connect or function since the Play app is available before Cassandra.

I've currently got the connector set up like this:

object Defaults {
  val connector = ContactPoint(sys.env("CASSANDRA_URL"), sys.env("CASSANDRA_PORT").toInt)
    .withClusterBuilder(_.withSocketOptions(
      new SocketOptions().setTcpNoDelay(true))
    ).keySpace("my_app")
}

With the database being initialized like this

class CassandraDB(val keyspace: KeySpaceDef) extends Database(keyspace) {
  object users extends ConcreteUsers with keyspace.Connector
  object articles extends ConcreteArticles with keyspace.Connector
  object comments extends ConcreteComments with keyspace.Connector
}


object CassandraDB extends CassandraDB(Defaults.connector)

And my Play! controller makes calls to the database using the CassandraDB Object

def index = Action.async {
  CassandraDB.users.getAll.map { users =>
    Ok(Json.toJson(users))
  }
}

The first attempt to connect to the database results in the expected NoHostAvailableException

com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042)

Any request after that will throw the following exception:

play.api.UnexpectedException: Unexpected exception[RuntimeException: java.lang.NoClassDefFoundError: Could not initialize class models.CassandraDB$]

Once that happens a manual restart of the application is required for it to work.

While waiting for the Cassandra container to fully initialize works just fine, this does not seem ideal and I was hoping to make it retry after it fails to connect

Answer 1

Well, it's clear to me that there is nothing wrong to phantom and/or your application right? Both works fine when the container is working properly.

Have you tried to order your compose?

https://docs.docker.com/compose/startup-order/

EDIT

Based on the OP question on my answer, a possible solution would be using an Actor approach, where you could use a supervisor strategy in case you database is out.

override def supervisorStrategy: SupervisorStrategy =
    OneForOneStrategy(maxNrOfRetries = 5) {
      case _: NoHostAvailableException => Restart
      case _: Exception => Stop
    }

So you could have an Actor that interacts with your database and when trying to connect, if a known error occurs, you can catch it on the supervisor and decides what to do. If you decide to restart, you could use the preRestart method to connect again.

override def preRestart(reason: Throwable, message: Option[Any]): Unit = {
    log.warning(s"Restarting Actor due: {}", reason.getMessage)
    //do something here
  }

http://doc.akka.io/docs/akka/2.4.11/general/supervision.html http://doc.akka.io/docs/akka/2.4.11/scala/fault-tolerance.html