Search code examples
cassandracolocation

Howt to achieve data locality on Cassandra node

I'm looking for a solution, that will allow me to colocate the code with the data. As a Db I have Cassandra and would like to be able to get the data, that is on a specific node.

The importance here is that I try to achieve it from my own code without using frameworks, as Hadoop or Spark.

I wounder, if someone could explain or provide a link, as I have not found yet a solution. The question here is, how that could be achieved with Cassandra.

Thanks in advance

Solution

  • Warning this is most likely not what you should be doing:

    The easiest way to do this would be to use the byte order partitioner. This places data on nodes based on the actual byte ordering of the primary key's rather than using a hash. This technique is for experts only and removes many of the benefits of Cassandra and only should be used by those who truly understand the tradeoffs. ByteOrderedPartitioner also places significantly more burden on the application designer and the sysops team because the system will now not be expandable in an easy to understand way.

    Using Spark or Hadoop is the correct way to deal with this:

    The real solution is to use Hadoop or Spark. You could get the data locality you are looking for by attempting to read sstables directly from the disk. An example of this: http://www.fullcontact.com/blog/cassandra-sstables-offline/