Is there any proper way to down a node when this become unreachable in akka cluster. I want to expose an api to down a node when it becomes unreachable but I prefer to find another way or something programmatically. Auto-downing is not an option anymore since I've been having split brain issues.
Can I just down the node in the receive method :
def receive: Receive = { case MemberUp(member) =>
case UnreachableMember(member) => {
Cluster(context.system).down(member.address)
}
If the node has crashed, downing it on any other node is fine.
But you must do it as a reaction to seeing it unreachable though, that will cause split brain if it is a network partition rather than a node crashing the unreachable each side of the partition will see the rest of the cluster as unreachable and down all those, and you will end up with two clusters thinking that they have downed the other side.
This is the exact reason why we have removed auto downing in Akka 2.6
If you want automatic removal of nodes with split brain resolution you will need to use/implement a downing provider that handles that for you.
Implementing a downing provider yourself will require quite a bit of effort and thinking it through. For more background on the problem you can read the docs of the commercial Lightbend SBR here: https://doc.akka.io/docs/akka-enhancements/current/split-brain-resolver.html