Search code examples
elasticsearchelasticsearch-plugin

Elasticsearch 2.2.0 date formats failed after upgrading from 1.5.2

Im trying to insert data to Elastisearch 2.2.0 with filed mapped as:

...
"startTime": {
   "format": "yyyy/MM/dd HH:mm:ss",
   "type": "date"
}
...

The data that inserted is (strings):

...
"startTime":1454858900301
...

I get exception:

MapperParsingException[failed to parse [startTime]]; nested: IllegalArgumentException[Invalid format: "1454858900301" is malformed at "0301"];
Caused by: java.lang.IllegalArgumentException: Invalid format: "1454858900301" is malformed at "0301"
    at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187)
    at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:780)
    at org.elasticsearch.index.mapper.core.DateFieldMapper$DateFieldType.parseStringValue(DateFieldMapper.java:362)
    at org.elasticsearch.index.mapper.core.DateFieldMapper.innerParseCreateField(DateFieldMapper.java:528)
    at org.elasticsearch.index.mapper.core.NumberFieldMapper.parseCreateField(NumberFieldMapper.java:241)
    at org.elasticsearch.index.mapper.FieldMapper.parse(FieldMapper.java:335)

It didn't happened in Elastic version 1.5.2 and before I upgraded to 2.2.0 I used the migration plugin that supposed to list out all mappings issues- nothing appeared...

What caused the exception? I inserting long epoch time string (in milliseconds) that suppose to be parsed to the format I listed in the mapping, no?

Why it worked in 1.5.2 and fail in 2.2.0?

Solution

  • I think you have to go through breaking change in date format in ES 2.X. From the docs,

    In 2.0, we have added two formats: epoch_millis and epoch_second. Only date fields that use these formats will be able to parse timestamps

    Migration tool won't detect this as this is not a mapping issue. As a resolution, you may have to change your date format to any of epoch_millis or epoch_second.