Search code examples
truncateaerospikeaerospike-ce

Is it possible to issue a durable delete in Aerospike with asinfo using 'truncate'?

I wanted to avoid using Aerospike clients (e.g. for Python) and delete records from a set using native asinfo command 'truncate' as it allows to do it quickly. But after I restarted Aerospike all deleted records were back. I saw this aerospike: delete all record in a set but it doesn't answer my question. Neither does this page from AS docs. It says, that a tombstone should be written after a durable delete, do I have to create it manually or are there some other ways?

UPD: Thanks to @kporter who provided the accepted answer below I was able to look into differences between Community and Enterprise edition of Aerospike and found more information on the problem, some may find it helpful as well:

If I understood all of it correctly the best way to get your records deleted completely in CE is to ensure that they have right TTL and can expire naturally. And if for some reason you have a lot of old records without TTL as in my case, you can issue truncate command via asinfo and do not restart AS server until data on SSD is eventually overwritten. Or just truncate sets with old records on every restart.

Also I wonder if it is possible to wipe AS storage completely and then restore it from a backup of already truncated data as an emergency measure?

UPD1: So, I was able to wipe SSD with Aerospike storage and restore only needed records from a backup. Here is how I did it:

  • Firstly, you need to remove old records from sets via asinfo and truncate command, links to docs are above
  • Then backup namespaces you want to save with asbackup
  • Stop your AS server, mine was in Docker container, so I just stopped said container
  • Zero out the disk that is used as AS storage, mine was /dev/sdb
  • Create necessary partitions on this disk
  • Start AS server
  • Restore data from the backup using arestore

Useful links: how to remove and clean up an aerospike server installation, AS docs on SSD setup

I am not sure if it is a good solution for large production setups but it worked as intended in my case with only one AS node and an opportunity to stop it for a while. This way I was able to reduce the size of data in my AS from 160Gb to 11Gb and because of that my server now fully restarts only in half an hour instead of approximately eight hours as before.

Solution

  • You can find more information about truncating a set here: https://www.aerospike.com/docs/operations/manage/sets/

    As mentioned there, truncation is not durable in Aerospike Community.

    In the Enterprise Edition, truncation is durable and preserves record deletions through a cold-restart. In the Community Edition, similar to record deletes, records in previously truncated sets are not durable and deletes can return through a cold-start.