Search code examples
elasticsearchelasticsearch-pluginjava

How can I set Ingest attachment(elasticsearch) plugin options with Java API?


I'm using Ingest Attachment Processor Plugin on elasticsearch. I need to set attachment options(indexed_chars, properties, ignore_missing etc.) with Java API. How can I do that?

I am creating index and setting pipeline like below:

String id = ...
Map<String, Object> row = ...
client.prepareIndex(indexName, "my_type", id)
                    .setSource(row)
                    .setPipeline("my_pipeline")
                    .execute();

Solution

  • I found answer, If you have nested document you must use foreach else build json like documentation

    Document:

    Solution:

    try (XContentBuilder jsonBuilder = XContentFactory.jsonBuilder()) {
        BytesReference pipelineSource = jsonBuilder.startObject()
                .field("description", "Extract attachment information")
                .startArray("processors")
                  .startObject()
                    .startObject("foreach")
                      .field("field", "my_field")
                      .startObject("processor")
                        .startObject("attachment")
                          .field("field", "_ingest._value.my_base64_field")
                          .field("target_field", "_ingest._value.my_base64_field")
                          .field("ignore_missing", true)
                          .field("indexed_chars", -1)
                        .endObject()
                      .endObject()
                    .endObject()
                  .endObject()
                .endArray()
                .endObject().bytes();
        client.admin().cluster().preparePutPipeline("my_pipeline",
                pipelineSource, XContentType.JSON).get();
    }
    

    OR

    you can put below json manually

    Result:

    http://localhost:9200/_ingest/pipeline/my_pipeline
    
    {
      "my_pipeline": {
        "description": "Extract attachment information",
        "processors": [
          {
            "foreach": {
              "field": "my_field",
              "processor": {
                "attachment": {
                  "field": "_ingest._value.my_base64_field",
                  "target_field": "_ingest._value.my_base64_field",
                  "ignore_missing": true,
                  "indexed_chars": -1
                }
              }
            }
          }
        ]
      }
    }