Search code examples
cassandranodetool

write audit in cassandra

Is there a way to find out the time when

  1. the last memtable flush/SS Table write happened - and what triggered it ie whether it was a memtable threshold breach, commitlog size breach,scheduled etc
  2. compaction and trigger for the same.

We need to benchmark our writes - keeping a record of intervening table writes, and compaction

Solution

  • The log file have those details. When and why of memtables, and when compactions run. But compactions just run whenever they can, like in size tiered whenever theres 4 or more similiarish sized sstables not really as a response to some event so the reason isn't logged, it just checks constantly for something to do.

    in debug.log:

    DEBUG [CompactionExecutor:2] 2018-08-31 00:53:27,376 CompactionTask.java:158 - Compacting (2e443b10-ace2-11e8-8bf9-5914dc23286d) [/Users/clohfink/git/cassandra/tmp/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/na-8-big-Data.db:level=0, /Users/clohfink/git/cassandra/tmp/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/na-5-big-Data.db:level=0, /Users/clohfink/git/cassandra/tmp/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/na-6-big-Data.db:level=0, /Users/clohfink/git/cassandra/tmp/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/na-7-big-Data.db:level=0, ]
DEBUG [CompactionExecutor:2] 2018-08-31 00:53:27,448 CompactionTask.java:256 - Compacted (2e443b10-ace2-11e8-8bf9-5914dc23286d) 4 sstables to [/Users/clohfink/git/cassandra/tmp/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6/na-9-big,] to level=0.  0.608KiB to 0.266KiB (~43% of original) in 65ms.  Read Throughput = 9.288KiB/s, Write Throughput = 4.055KiB/s, Row Throughput = ~12/s.  8 total partitions merged to 6.  Partition merge counts were {1:5, 3:1, }  

DEBUG [PerDiskMemtableFlushWriter_0:4] 2018-08-31 00:59:30,493 Memtable.java:453 - Writing Memtable-prepared_statements@841944213(0.474KiB serialized bytes, 4 ops, 0%/0% of on/off-heap limit), flushed range = (min(-9223372036854775808), max(9223372036854775807)]                                                  
DEBUG [PerDiskMemtableFlushWriter_0:4] 2018-08-31 00:59:30,493 Memtable.java:483 - Completed flushing /Users/clohfink/git/cassandra/tmp/data/system/prepared_statements-18a9c2576a0c3841ba718cd529849fef/na-2-big-Data.db (0.392KiB) for commitlog position CommitLogPosition(segmentId=1535694804718, position=5997190)