All messages receive a "user level notice"

Question

Im trying to parse a message from my network devices which send messages in format similar to

<30>Feb 14 11:33:59 wireless: ath0     Sending auth to xx:xx:xx:xx:xx:xx. Status: The request has been declined due to MAC ACL (52).\n
<190>Feb 14 11:01:29 CCR00 user admin logged out from xx.xx.xx.xx via winbox
<134>2023 Feb 14 11:00:33 ZTE command-log:An alarm 36609 level notification occurred at 11:00:33 02/14/2023 CET sent by MCP GponRm notify: <gpon-onu_1/1/1:1> SubType:1 Pos:1 ONU Uni lan los. restore\n on  \n

using this logstash.conf file

input {
    beats {
        port => 5044
    }

    tcp {
        port => 50000
    }
  udp {
        port => 50000
    }
}

## Add your filters / logstash plugins configuration here

filter {

      grok {    
        match => { 
        "message" => "^(?:<%{POSINT:syslog_pri}>)?%{GREEDYDATA:message_payload}"
       }
     }   
      syslog_pri {    
         }
      mutate {
    remove_field => [ "@version" , "message" ]
     }
}


output {
    stdout {}
    elasticsearch {
        hosts => "elasticsearch:9200"
        user => "logstash_internal"
        password => "${LOGSTASH_INTERNAL_PASSWORD}"
    }
}

which results in this output

{
  "@timestamp": [
    "2023-02-14T10:38:59.228Z"
  ],
  "data_stream.dataset": [
    "generic"
  ],
  "data_stream.namespace": [
    "default"
  ],
  "data_stream.type": [
    "logs"
  ],
  "event.original": [
    "<14> Feb 14 11:38:59 UBNT BOXSERV[boxs Req]: boxs.c(691) 55381193 %% Error 17 occurred reading thermal sensor 2 data\n\u0000"
  ],
  "host.ip": [
    "10.125.132.10"
  ],
  "log.syslog.facility.code": [
    1
  ],
  "log.syslog.facility.name": [
    "user-level"
  ],
  "log.syslog.severity.code": [
    5
  ],
  "log.syslog.severity.name": [
    "notice"
  ],
  "message_payload": [
    " Feb 14 11:38:59 UBNT[boxs Req]: boxs.c(691) 55381193 %% Error 17 occurred reading thermal sensor 2 data\n\u0000"
  ],
  "syslog_pri": [
    "14"
  ],
  "_id": "UzmBT4YBAZPdbqc4m_IB",
  "_index": ".ds-logs-generic-default-2023.02.04-000001",
  "_score": null
}

which is mostly satisfactory, but i would expect the log.syslog.facility.name and log.syslog.severity.name fields to be processed by the syslog_pri filter with imput of <14> to result into secur/auth and Alert recpectively,

but i keep getting the default user-level notice for all my messages, no matter what the part of the syslog message contains

anyone could advise and maybe fix my .conf syntax, if its wrong?

thank you very much!

i have logstash configured properly to receive logs and send them to elastics, but the grok/syslog_pri doesnt yield expected results

Answer 1

The fact that the syslog_pri filter is setting [log][syslog][facility][code] shows that it has ECS compatibility enabled. As a result, if you do not set the syslog_pri_field_name option on the syslog_pri filter, it will try to parse [log][syslog][priority]. If that field does not exist then it will parse the default value of 13, which is user-level/notice.