Search code examples
elasticsearchttlgraylog2

elasticsearch ttl force delete old messages


Im set TTL as in manual, but elasticsearch documentation said that messages older then TTL time will be not affected, how can i force deletion of older messages?

Or maybe exists some method to delete messages older then 30 days?


Solution

  • Following from our comment back-and-forth:

    I can't access that doc since it's local to your computer. But from the info is seems you're referring to a default of 30 days, probably set by graylog2 (I'm sure it isn't set By Elasticsearch).

    But to answer your question: if you've got a timestamp field on your documents do a delete-by-query on that timestamp (select all docs older than 30 days and delete). http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-delete-by-query.html.

    To be save, be sure to do a fetch first on the same query instead of a delete to see if you've actually formulated the query correctly.

    If you don't have a timestamp on your documents, I'm really not sure how to do this. Still I like to see evidence/docs that Elasticsearch doesn't delete these according to the TTL.