Search code examples
cassandra

How does Cassandra optimize storage internally if each row can have different columns?


For example Column Family Row 1 -> (col1, col2, col3) Row 2 -> (col1, col4, col5)

Each column can be a completely different type with different length, making byte alignment difficult, how does cassandra optimize its store internally?


Solution

  • Cassandra implements sparse storage so unlike traditional databases, only the columns which contain values are stored on disk. And with the exception of a few CQL data types like integers, most types don't have a fixed size so the stored rows can grow dynamically.

    To use your example, the on-disk representation of the first row would look like:

    | pk = key1 | col1 = 1 | col2 = 2 | col3 = 3 |
    

    The second row would look like:

    | pk = key2 | col1 = 1 | col4 = 4 | col5 = 5 |
    

    Notice that for the first row, columns col4 and col5 are not stored on disk. Similarly for the second row, columns col2 and col3 are not stored.

    Cassandra stores metadata about the partitions including sizes and corresponding offsets so it knows the start/end boundaries of the data, including where in the SSTable (data file) each partition exists which makes reading faster since it saves from having to read from the start of each SSTable.

    These don't directly address your questions but if you're interested, have a look at How data is written and How data is read in Cassandra. Cheers!