So I'm trying to setup multi-node cassandra cluster.
I've setup 3 cassandra nodes on 3 different servers. In all of the three cassandra.yaml files, I've defined the same one seed node. After clearing the data and restarting the services one by one, I can see them showing up and the nodetool status shows UN for all entries. Now I use express-cassandra for connecting to the cassandra, and that works just fine, but it only works for the first few queries and afterwards, it just crashes with the following error -
Error during find query on DB -> ResponseError: Server failure during read query at consistency ONE (1 responses were required but only 0 replicas responded, 2 failed)
It does not matter on which read query it fails, but it's up for like 10 seconds and then it dies. Sometimes it dies on queries which were successful on previous run. Also, I'm just wondering, why it says that 2 failed, if there are 3 nodes (one seed node)?
I noticed, that express-cassandra notified that the Replication factor has been changed and to run the nodetool repair, but there I keep getting the following error message -
Validation failed in /xxx.xxx.xxx.xxx (progress: 0%)
[2018-07-02 08:10:33,447] Some repair failed
Where xxx.xxx.xxx.xxx is one of the three node IP addresses. I tried to run the nodetool repair on each of the nodes, but I keep getting identical errors on each of the servers.
These are my express-cassandra setup properties (xxx.xxx.xxx.xxx is my seed ip address):
clientOptions: {
contactPoints: ['xxx.xxx.xxx.xxx'],
protocolOptions: {
port: 9042
},
keyspace: 'test_keyspace',
queryOptions: {
consistency: cassandra.consistencies.one
},
socketOptions: {
readTimeout: 0
}
},
ormOptions: {
defaultReplicationStrategy: {
class: 'NetworkTopologyStrategy',
dc1: 3
}
}
I'm not sure if the NetworkTOpologyStrategy is correctly set up, but after starting up the express.js I ran a write query to the database, and it populated the data on all three nodes.
Any help on resolving these both errors would be wonderful!
Providing an answer here, just in case anyone else struggles.
The issue was related to the NetworkTopologyStrategy. I switched to SimpleStrategy, and it started to work.
I had just one DC.
Not sure, what exactly was causing the issue, but if anyone is aware, feel free to add it in comment, and I'll update the answer.