Configure elastic pipeline attachment in logstash configuration?

Question

I'm using ingest-attachment to get data of base64 field.

PUT _ingest/pipeline/attachment
{
  "description" : "Extract attachment information",
  "processors" : [
    {
      "attachment" : {
        "field" : "data"
      }
    }
  ]
}



PUT my_index/my_type/my_id?pipeline=attachment
{ 
  "data": "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0="
}

GET my_index/my_type/my_id

{
  "found": true,
  "_index": "my_index",
  "_type": "my_type",
  "_id": "my_id",
  "_version": 1,
  "_source": {
    "data":     "e1xydGYxXGFuc2kNCkxvcmVtIGlwc3VtIGRvbG9yIHNpdCBhbWV0DQpccGFyIH0=",
"attachment": {
  "content_type": "application/rtf",
  "language": "ro",
  "content": "Lorem ipsum dolor sit amet",
  "content_length": 28
   }
  }
}

How do I do the same thing using Logstash configuration file? I have not seen any reference to define pipeline attachment.

Answer 1

You simply need to specify which pipeline to use in your elasticsearch output configuration, so that your event gets processed by the proper pipeline, like this:

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "my_index"
        document_type => "my_type"
        pipeline => "attachment"              <--- use this
    }
}

PS: note that this will only work on ES 5 and Logstash 5.