Search code examples
elasticsearchlogstash

how to add auto remove field in logstash filter

I am trying to add a _ttl field in logstash so that elasticsearch removes the document after a while, 120 seconds in this case but that's for testing.

filter {
    if "drop" in [message] {
        drop { }
    }

    add_field => { "_ttl" => "120s" }
}

but now nothing is logged in elasticsearch.

I have 2 questions. Where is logged what is going wrong, maybe the syntax of the filter is wrong?

How do I add a ttl field to elasticsearch for auto removal?

Solution

  • When you add a filter to logstash.conf with a mutator it works:

    filter {
    mutate {
        add_field => { "_ttl" => "120s" }
    }
}

POST myindex/_search
{
  "query": {
    "match_all": {}
  }
}

    Results:

    "hits": [
  {
    "_index": "myindex",
      ...................
      "_ttl": "120s",

    For the other question, cant really help there. Im running logstash as container so logging is read with:

    docker logs d492eb3c3d0d