Search code examples
kibanakibana-5

Is Kibana date format index pattern still supported?

I got latest Kibana 5.4.0 and Docs says:

https://www.elastic.co/guide/en/kibana/current/index-patterns.html#settings-create-pattern

To use an event time in an index name, enclose the static text in the pattern and specify the date format using the tokens described in the following table.

For example, [logstash-]YYYY.MM.DD matches all indices whose names have a timestamp of the form YYYY.MM.DD appended to the prefix logstash-, such as logstash-2015.01.31 and logstash-2015-02-01.

When I try to create pattern [testx_]YYYY-MM-DD_HH-mm or [testx_]YYYY-MM-DD_HH or [testx_]YYYY-MM-DD Kibana can't find @timstamp field and says that none of indexes match these patterns.

GET _cat/indices
yellow open testx_2017-06-19_14    dHAfSzAuSEKpYLuA8p5EIw 1 1 1 0 4.6kb 4.6k
yellow open testx_2017-06-19_13-59 hfGkELCsSUavaX8GuLPuMQ 1 1 1 0 4.6kb 4.6kb
yellow open testx_2017-06-19       lbsdW18cSIuZ2bNn1Fw7WA 1 1 1 0 4.6kb 4.6kb

On other hand for testx_* pattern Kibana finds @timestamp field and matches 100% of indexes...

Do latest Kibana support time based names for indexes?

I would like to gain performance benefits from index naming schema if it's still appropriate...

UPDATE

What is wrong: failed

Some warnings: some warnings

UPDATE 2 I found https://www.elastic.co/blog/managing-time-based-indices-efficiently which promote "Rollover Pattern". Maintaining date/time in index name is no longer a recommended way, but I doubt that new API makes life easier ((

Solution

  • According to these issues:

    Kibana 4.3.0 should address this for you: it automatically optimizes wildcard index patterns such as logstash-* in the same way that you could previously only achieve by manually configuring a time-based index pattern name that matches your underlying indexing scheme (e.g. [logstash-]YYYY.MM.DD).

    Elasticsearch 1.6 introduced the _field_stats API which will, for the first time, allow us to search for indices that contain fields within a given range. For example, we can search for indices that contain an @timestamp between X and Y.

    This means that users will no longer be required to roll their indices at UTC midnight, nor use date patterns at all. They can effectively name indices whatever they want. and Kibana can automatically optimize requests by firing a pre-flight request for indices. We might need to add some caching here, but it should greatly enhance usability.

    There is no need for time based names for performance but keeping time based index names still useful for archiving old indexes.

    UPDATE Created issue to remove time based pattern from docs. https://github.com/elastic/kibana/issues/12406