Email notification using Logstash

Question

I am using ELK version 8.10.2-1 for Kong API Gateway logs. I would like to configure Logstash email plugin to send email notification based on the HTTP status code.

In ELK Kong Logs, I have a field called [response.status] which contains HTTP status codes such as 200, 401, and 500.

I have configured Logstash to add a custom tag based on the value of [response.status] field, but unfortunately the tag is not working successfully.

How to configure/add tags based on the value of the [response.status] field?

Logstash configuration file:

input {
  udp {
    port => 9000
  }
}

filter {
  json {
    source => "message"

    add_tag => ["kong"]

  }


##
if [response.status] == "401" {
    mutate {
      add_tag => [ "http_error_401" ]
    }
  }
##

}

output {
  elasticsearch {
    hosts => ["https://xx.xx.xx.xx:9200" , "https://xx.xx.xx.xx:9200" , "https://xx.xx.xx.xx:9200"]
    user => "elastic_user"
    password => "${elastic_password}"
    ssl => true
    ssl_certificate_verification => false
    cacert => "/etc/logstash/http_ca.crt"
    ilm_rollover_alias => "kong"
    ilm_pattern => "{now/d}-000001"
    ilm_policy => "kong-index-policy-example"

  }
if "http_error_401" in [tags] {
    email {
      to => "[email protected]"
      from => "[email protected]"
      subject => "API-Error at  %{@timestamp}"
      body => "Tags: %{tags}\\n\\Content:\\n%{message}"
      via => "smtp"
      address => "mail.example.com"
      port => 25

    }
  }
}

Answer 1

Solution: update if [response.status] == "401" to if [response][status] == "401" or/and if [response][status] == 401

Here is the explanation:

[response.status]

The syntax if [response.status] == "401" in Logstash is incorrect because it attempts to reference a nested field using dot notation. Logstash primarily uses square bracket notation for nested fields. [response.status]: This is interpreted as a single field named "response.status" rather than as nested fields "response" and "status."

[response][status] The syntax [response][status] in Logstash is known as field reference syntax. It is used to access nested fields within the event data. By combining them with [response][status], you are specifying the full path to the nested field "status" within the "response" field. This syntax allows Logstash to correctly interpret and access nested fields in your conditional statements or other processing logic.

Here is the official documentation if you want to check: https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html

Here is an example if you want to test:

#logstash conf:

input {
  stdin {codec => json}
}
filter {
  if [response][status] == 401 {
    mutate {
      add_tag => [ "http_error_401" ]
    }
  }
}
output {
  if "http_error_401" in [tags] {
  stdout {}
  }
}

#data:

{"response":{"status":401}}

#run logstash

./logstash-7.16.2/bin/logstash -f email.conf

#output:

{"response":{"status":401}}
{
          "host" => "musab-mac.local",
          "tags" => [
        [0] "http_error_401"
    ],
      "response" => {
        "status" => 401
    },
      "@version" => "1",
    "@timestamp" => 2023-12-18T14:27:40.534Z
}