RemoteTransportException, Fielddata is disabled on text fields when doing aggregation on text field

Question

I am migrating from 2.x to 5.x I am adding values to the index like this

indexInto (indexName / indexType) id someKey source foo

however I would also want to fetch all values by field:

 def getValues(tag: String) ={
client execute {
search(indexName / indexType) query ("_field_names", tag) aggregations (termsAggregation( "agg") field tag size 1)
}

But I am getting this exception :

RemoteTransportException[[8vWOLB2][172.17.0.5:9300][indices:data/read/search[phase/query]]]; nested: IllegalArgumentException[Fielddata is disabled on text fields by default. Set fielddata=true on [my_tag] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.];

I am thought maybe to use keyword as shown here , but the fields are not known in advanced (sent by the user) so I cannot use perpend mappings

Answer 1

By default all the unknown fields will be indexed/added to elasticsearch as text fields which are not specified in the mappings. If you will take a look at mappings of such a field, you can see there a field is enabled with for such fields with type 'keyword' and these fields are indexed but not analyzed.

GET new_index2/_mappings
{
  "new_index2": {
    "mappings": {
      "type": {
        "properties": {
          "name": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}

so you can use the fields values for the text fields for aggregations like the following

POST new_index2/_search
{
  "aggs": {
    "NAME": {
      "terms": {
        "field": "name.fields",
        "size": 10
      }
    }
  }
}

Check name.fields

So your scala query can work if you can shift to fields value.

def getValues(tag: String) = {
   client.execute {
      search(indexName / indexType)
        .query("_field_name", tag)
        .aggregations {
           termsAgg("agg", "field_name.fields")
        }.size(1)
   }
}

Hope this helps. Thanks