I'm testing the behaviour of a Cassandra cluster with different configurations. One of them uses a ByteOrderedPartitioner to distribute the keys along the nodes.
The data model is made by fat rows, each of them with a integer (IntegerType) as key with a value from 0 to 9999. I setted a ring of 10 nodes in such way:
Address Rack Status State Owns Token
Token(bytes[0900])
node1 Up Normal 5.14 MB 10.00% Token(bytes[00])
node2 Up Normal 377.42 MB 10.00% Token(bytes[0100])
node3 Up Normal 373.06 MB 10.00% Token(bytes[0200])
node4 Up Normal 966.15 KB 10.00% Token(bytes[0300])
node5 Up Normal 953.4 KB 10.00% Token(bytes[0400])
node6 Up Normal 953.26 KB 10.00% Token(bytes[0500])
node7 Up Normal 944.54 KB 10.00% Token(bytes[0600])
node8 Up Normal 953.4 KB 10.00% Token(bytes[0700])
node9 Up Normal 962.42 KB 10.00% Token(bytes[0800])
node10 Up Normal 961.31 KB 10.00% Token(bytes[0900])
but all the load goes on the node2 and on the node3 for replica.
Why these tokens don't work? How I should fix them?
First, you should run cleanup on each node to make sure you're not seeing extra data that the node is no longer responsible for, post-bootstrap.
Once that is done, you can use the getRangeKeySample JMX method to get a sample of the keys on a node. You can then adjust its token to a value that moves an appropriate amount of keys to its neighbor, if necessary.
That said: you should almost certainly be using RandomPartitioner instead, to avoid exactly this problem. I've explained in posts like this how you can use clustering within a partition key to do the kind of ordering you might otherwise be tempted to use BOP for.