Add documents to elasticsearch if it does not exists

Question

I transferred some data from a log generated every day to elasticsearch using logstash, and my logstash output section looks like :

i keep the same id (id_ot) in both my log file and elasticsearch, but what i would like to do is : if the new coming id ( id_ot) already exists in elasticseach, so i will not insert it. How can i do that in logstash ?

Any help would be really appriciated !

Answer 1

You simply need to add the action => create parameter and if a document already exists with that ID, it is not indexed

output {
  elasticsearch {
    ...
    action => "create"
  }
}