How to create actors transparently across an Akka cluster

I have an actor system shown in the following graph.

Master
|
|--Monitor
|
|--Supervisor
      |
      |--Device #1
      |--Device #2
      |-- ...
      |--Device #N

The Master actor starts the Monitor actor and the Supervisor actor.

The Monitor actor fires a message telling itself to check the device inventory every 30 seconds. For each device that exists in the inventory, it sends a message to the Supervisor actor to register the device.

The Supervisor actor starts the device actor when it receives the message from the Monitor actor to register the device.

The code is available here on github.

Everything works pretty well in a single JVM instance, but when it comes to cluster mode, I start to be confused.

As I expected, with the following configuration, the previous actor tree should run transparently in the cluster machines. All the actors should be created across the cluster, and I should not care on which machine the actors are created.

On instance #1:

akka {
  actor {
    provider = "cluster"
  }
  remote {
    log-remote-lifecycle-events = off
    netty.tcp {
      hostname = "127.0.0.1"
      port = 2551
    }
  }

  cluster {
    seed-nodes = [
      "akka.tcp://example@127.0.0.1:2551",
    ]
  }
}

On instance #2:

akka {
  actor {
    provider = "cluster"
  }
  remote {
    log-remote-lifecycle-events = off
    netty.tcp {
      hostname = "127.0.0.1"
      port = 2552
    }
  }

  cluster {
    seed-nodes = [
      "akka.tcp://example@127.0.0.1:2551",
    ]
  }
}

What I expected:

Master (On node 1)
|
|--Monitor (On node 2)
|
|--Supervisor (On node 2)
      |
      |--Device #1 (On node 1)
      |--Device #2 (On node 1)
      |-- ...
      |--Device #N (On node 2)

But, it ends up running two identical actor trees in the cluster: each node runs one actor tree separately as the following graph shows:

Node 1             |   Node2
                   | 
Master             |   Master
|                  |   |
|--Monitor         |   |--Monitor  
|                  |   | 
|--Supervisor      |   |--Supervisor 
    |              |       |      
    |--Device #1   |       |--Device #1
    |--Device #2   |       |--Device #2
    |-- ...        |       |-- ... 
    |--Device #N   |       |--Device #N

I also tried to run the Master actor as a singleton actor in the cluster, but it ends up running a single actor tree on the very node on which the Master is started.

So, my question is, how could I create an actor across the cluster by calling context.actorOf(DeviceActor.props(deviceId)), without carefully designing the actor tree with cluster singleton and cluster sharding?

Solution

What you describe is the exact definition of Akka cluster sharding.

Cluster sharding is useful when you need to distribute actors across several nodes in the cluster and want to be able to interact with them using their logical identifier, but without having to care about their physical location in the cluster, which might also change over time.

It's pretty simple to set it up, especially if you already have a cluster. Monitor would be a Cluster Singleton so that there's only one alive on the cluster, and each Device would be an entity identified by the device ID.