Search code examples
elasticsearchkibanametricbeat

Kibana scripted field which loops through an array

I am trying to use the metricbeat http module to monitor F5 pools.

I make a request to the f5 api and bring back json, which is saved to kibana. But the json contains an array of pool members and I want to count the number which are up.

The advice seems to be that this can be done with a scripted field. However, I can't get the script to retrieve the array. eg

doc['http.f5pools.items.monitor'].value.length()

returns in the preview results with the same 'Additional Field' added for comparison:

[
 {
  "_id": "rT7wdGsBXQSGm_pQoH6Y",
  "http": {
   "f5pools": {
    "items": [
     {
      "monitor": "default"
     },
     {
      "monitor": "default"
     }
    ]
   }
  },
  "pool.MemberCount": [
   7
  ]
 },

If I try

doc['http.f5pools.items']

Or similar I just get an error:

"reason": "No field found for [http.f5pools.items] in mapping with types []"

Googling suggests that the doc construct does not contain arrays?

  1. Is it possible to make a scripted field which can access the set of values? ie is my code or the way I'm indexing the data wrong.
  2. If not is there an alternative approach within metricbeats? I don't want to have to make a whole new api to do the calculation and add a separate field

-- update.

Weirdly it seems that the number values in the array do return the expected results. ie.

doc['http.f5pools.items.ratio']

returns

 {
  "_id": "BT6WdWsBXQSGm_pQBbCa",
  "pool.MemberCount": [
   1,
   1
  ]
 },

-- update 2

Ok, so if the strings in the field have different values then you get all the values. if they are the same you just get one. wtf?

Solution

  • OK, solved it.

    https://discuss.elastic.co/t/problem-looping-through-array-in-each-doc-with-painless/90648

    So as I discovered arrays are prefiltered to only return distinct values (except in the case of ints apparently?)

    The solution is to use params._source instead of doc[]