Search code examples
elasticsearchaws-elasticsearchelasticsearch-opendistro

How to get variable values in triggered alert in AWS ElasticSearch Service?


I'm trying to post detailed message on Slack from AWS ElasticSearchService Monitor Alert Trigger. I've tried to use variables as in description https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/monitors/

Alerts are getting triggered and posted on Slack, but most of variables are empty, when I try to use them in action. I get information only from ctx.trigger.name, ctx.periodStart, ctx.periodEnd. Variables like ctx.trigger.condition, ctx.results[0], ctx.error, ctx.results[0].hits.total are empty, while the docs are saying ctx.error will be filled, if ctx.results[0] is empty.

How do I get more data? Is my query limiting it somehow?

My Monitor extraction query is following:

{
    "size": 20,
    "query": {
        "constant_score": {
            "filter": {
                "bool": {
                    "must": [
                        {
                            "range": {
                                "@timestamp": {
                                    "from": "now-1m",
                                    "to": null,
                                    "include_lower": true,
                                    "include_upper": true,
                                    "boost": 1
                                }
                            }
                        }
                    ],
                    "must_not": [
                        {
                            "match": {
                                "status": {
                                    "query": "200",
                                    "operator": "OR",
                                    "prefix_length": 0,
                                    "max_expansions": 50,
                                    "fuzzy_transpositions": true,
                                    "lenient": false,
                                    "zero_terms_query": "NONE",
                                    "auto_generate_synonyms_phrase_query": true,
                                    "boost": 1
                                }
                            }
                        }
                    ],
                    "adjust_pure_negative": true,
                    "boost": 1
                }
            },
            "boost": 1
        }
    }
}

Solution

  • The trigger syntax uses Moustache templates (man page), which is not comprehensive. The trick is to loop through the results and refer to variables like this:

    - Total hits: {{#ctx.results}}{{#hits}}{{total}}{{/hits}}{{/ctx.results}}
    - Period start: {{ctx.periodStart}}
    - Period end: {{ctx.periodEnd}}
    - HTTP errors:
      {{#ctx.results}}
          {{#hits}}
              {{#hits}}
                  {{#_source}} ip:{{ip}} status: {{status}} : {{error.message}} at path: {{path}} {{/_source}}
              {{/hits}}
          {{/hits}}
      {{/ctx.results}}