Search code examples
scalasinglestore

How to free space for dropped database in MemSql

I am new to MemSQL. I have created a database and tables in MemSQL on cluster with 5 leaf nodes and 2 aggregator nodes. Spark is running on the same cluster. Everything is in default mode. Inserted data and deleted the same. select * does not return anything. But when I saw web clusterUI, each leaf nodes are still consuming around 6TB of disk space.

enter image description here enter image description here Disk Capacity description says "This is is amount of disk space used by MemSQL relative to total disk space available. When this is full, no snapshot, transaction logs or columnstore data can be created".

With this description, I assume that above 6TB disk space is because of MemSQL usage.

Can some please clarify

  1. Does by default MemSQL write data to disk too ?
  2. Does is not delete the data written on disk even after deleting content and dropping table?
  3. How can I free disk space consumed by MemSQL? Which directory I hvae to delete? I figured out that directory "/var/lib/memsql/leaf-3306" has all memsql artifacts for this leaf node.
Solution

  • The 6TB usage might be because of MemSQL, or it might be because of Spark or some other process. MemSQL ops reports total disk usage, not disk used by MemSQL (the tooltip is slightly misleading).

    1) Rowstore tables (tables WITHOUT a CLUSTERD COLUMNSTORE index) write logs to disk for every write. The logs are combined into snapshots when the logs become to large, and by default, we keep the last two snapshot files. Thus, it is possible that the older of the two snapshots contains the data you deleted. You can trigger a new snapshot with SNAPSHOT <dbName>, and this will let GC cleanup the old (possibly large) ones.

    2) Snapshots and logs are per database, not per table. Dropping a table will not trigger snapshot/log cleanup, but dropping the database or triggering a new snapshot will.

    3) You probably shouldn't delete data directories by hand. DROP DATABASE <db_name> will delete all data associated with that database.

    For columnstore tables, the story is slightly different, but I assume "Everything is default" means no columnstore tables.