Search code examples
elasticsearchcassandraelassandra

Cassandra new node unable to join the cluster


Actually I'm adding new Elassandra node but the issue obviously involved to the Cassandra configurations.

Previously, I have two nodes in the cluster. There're perfectly connect with each other. Using nodetool status also returns both nodes information.

Then I added the new node to the cluster by followed this article https://docs.datastax.com/en/archived/cassandra/2.0/cassandra/operations/ops_add_node_to_cluster_t.html

the parameters I've set in the new node,

cluster_name, 
seeds (without itself), 
listen_address (its ip), 
start_native_transport: true, 
endpoint_snitch: GossipingPropertyFileSnitch, 
rpc_address (its ip)

the service fails with logs

2020-08-05 03:32:53,815 ERROR [main] ElassandraDaemon.java:585 main Exception
java.lang.RuntimeException: Unable to gossip with any peers
    at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1435)

then I changed the configuration by adding new node ip to seeds. the service is working fine but the nodetool status (on new node) show only itself and no ERROR in /var/cassandra/system.log. In the other hands, the new node is not visible in the existed two nodes.


Solution

  • I've seen this quite a bit and it's usually environmental rather than a Cassandra issue.

    Check that there's bi-directional traffic between nodes on the listen_address IPs and that they can contact each other on gossip port 7000.

    You mostly want to eliminate software/hardware firewalls blocking the traffic. Otherwise, ensure that network connectivity exists.

    I've discussed this in detail in this post if it helps you -- https://community.datastax.com/questions/7413/. Cheers!