I have a couchbase bucket which has a number of documents. Over a period of time , I see that these documents are rapidly taking up a lot of storage space. I am now working on setting TTL for all the new documents that will be stored. Is there way to set TTL for all the existing document or delete the existing documents based on expiry time? Different documents have different expiry time based on the document type (ranging from 15 minutes to 1 month). Please can you suggest an approach that I can use?
You can set the Expiry on a document and then update that document. Of course, you'd have to go through all the documents and set the Expiry for each.
I don't know how to do this in Java, but it's probably similar to .NET:
// get the document into a variable named 'doc', then
doc.Expiry = 123;
_bucket.Update(doc);
If you only have a few well-known documents, then this should be easy.
You can also use a N1QL query to retrieve documents based on expiry time. See this blog post for more information, but the gist is a query like this:
SELECT META(default).id, *
FROM default
WHERE DATE_DIFF_STR(STR_TO_UTC(exp_datetime),MILLIS_TO_UTC(DATE_ADD_MILLIS(NOW_MILLIS(),30,"second")),"second") < 30
AND STR_TO_UTC(exp_datetime) IS NOT MISSING;
Which would select documents that will expire in less than 30 seconds. So you could write a N1QL DELETE query that uses a WHERE clause.
UPDATE: A coworker at Couchbase pointed me to issue MB-16242. You can't set the expiry with a N1QL UPDATE yet. But as I said above, you can SELECT/DELETE documents based on the expiry.