Search code examples
elasticsearchelasticsearch-dsl

Elasticsearch 8 similarity module for date fields


After upgrading elasticsearch to 8.13.0 I noticed that similarity module no longer works with fields other than TextField and KeywordField: https://www.elastic.co/guide/en/elasticsearch/reference/current/similarity.html

Earlier I used similarity: boolean and date fields and it worked:

birth_date = fields.DateField(similarity="boolean")

but right now it throws error:

BadRequestError(400, 'mapper_parsing_exception', 'unknown parameter [similarity] on mapper [birth_date] of type [date]')

My question is: how to use similarity for scoring date fields in newer version of ES?


Solution

  • My hunch is that you were previously on version 7.x and upgraded to 8.13. You should have seen in your 7.x deprecation log a warning looking like this:

    Parameter [similarity] is deprecated and will be removed in a future version

    Looking into the source code of FieldMapper, we can see the following comment:

    // These parameters were previously *always* parsed by TypeParsers#parseField(), even if they
    // made no sense; if we've got here, that means that they're not declared on a current mapper,
    // and so we emit a deprecation warning rather than failing a previously working mapping.
    private static final Set<String> DEPRECATED_PARAMS = Set.of("store", "meta", "index", "doc_values", "index_options", "similarity");
    

    This basically means that it never made sense to specify those parameters, among which similarity, but for some reasons the parsers were lenient enough to let them go through.

    You can simply remove that similarity parameter as it doesn't serve any purpose on date fields anyway.