Search code examples
kibanaquerydslkibana-4

Kibana dashboard - source data from 2 indices with the same mapping


I am trying to setup Kibana dashboard. In Elasticsearch, there is one type of document only.
We have two indices with the same mapping:

Indices product_1 and product_2:

"_index": "product_1",
"_type": "product",
"_id": "3da33451d10f095c4b8cd485133dc23639244538",
"_score": 1,
"_source": {
    ...
    source": { "brand": "","category": "","store": "Store1"}
    ...
}

"_index": "product_1",
"_type": "product",
"_id": "3da33451d10f095c4b8cd485133dc23639244538",
"_score": 1,
"_source": {
    ...
    source": { "brand": "","category": "","store": "Store2"}
    ...
}
...

We have managed to create follow visualization.
We need to count total quantity of products (documents) per field store:

Top 0 source.store    Count
Store1                52,517
Store2                31,517
Store3                12,838
...

So, question is: How do we add data from second index here, in order to get result like that:

Top 0 source.store    Count (product_1)    Count (product_2)
Store1                52,517               42,517
Store2                31,517               56,517
Store3                12,838               13,890
...

Thanks a lot.

PS: We have managed to collect data from 2 indices:
["product_1","product_2"]
So kibana sources data, as if it is one index, and we get follow wrong result:

Top 0 source.store    Count (product_1+product_2)
Store1                102,517
Store2                62,517
Store3                24,838
...

Solution

  • I am not entirely sure if what you are trying is even possible. From the elastic search index documentation:

    The easiest and most familiar layout clones what you would expect from a relational database. 
    You can (very roughly) think of an index like a database.
    
    MySQL => Databases => Tables => Columns/Rows
    Elasticsearch => Indices => Types => Documents with Properties
    

    And as I understand here, you are trying to get data from two databases(indices) in a single query, which to the best of my knowledge, is not possible - at least the way you want it, i.e. assort results according to indices.

    Now you might (should) have a question that if that is entirely impossible, how did you get the following results?

    Top 0 source.store    Count (product_1+product_2)
    Store1                102,517
    Store2                62,517
    Store3                24,838
    

    This is where the Kibana magic lies. You can create an index pattern with a wild card. That will query every index that falls within the card. For example creating logstash-* will query all the logstash-[date] indices and give out consolidated results that you see. In your case it can be achieved by creating an index pattern as product_*. But I think you cannot get index wise assorted results with that (which I suppose, is exactly what you're looking for).