rsyslog is not forwarding logs to elasticsearch

Question

I'm trying to configure rsyslog to send logs to logstash and then forward them to elasticsearch.

I have create a config file /etc/rsyslog.d/60-output.conf with the following content:

*.* @localhost:10514;json-template

and a template file /etc/rsyslog.d/01-json-template.conf with the following content:

template(name="json-template"
  type="list") {
    constant(value="{")
      constant(value="\"@timestamp\":\"")     property(name="timereported" dateFormat="rfc3339")
      constant(value="\",\"@version\":\"1")
      constant(value="\",\"message\":\"")     property(name="msg" format="json")
      constant(value="\",\"sysloghost\":\"")  property(name="hostname")
      constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")
      constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")
      constant(value="\",\"programname\":\"") property(name="programname")
      constant(value="\",\"procid\":\"")      property(name="procid")
    constant(value="\"}\n")
}

Then I restarted rsyslog service. And for logstash I created a config file /etc/logstash/conf.d/logstash.conf with the following content:

input {
  udp {
    port => 10514
    codec => "json"
    type => "rsyslog"
  }
}
filter { }
output {
  if [type] == "rsyslog" {
    elasticsearch {
      hosts => [ "localhost:9200" ]
    }
  }
}

Then I restarted logstash.

When I run sudo netstat -tulpn | grep 10514 I get this:

user@rsyslog-server:/var/log$ sudo netstat -tulpn | grep 10514
udp        0      0 0.0.0.0:10514           0.0.0.0:*                           5327/java 

so Logstash is listening on port 10514.

To verify the elasticsearch input I run curl -XGET 'http://localhost:9200/logstash-*/_search?q=*&pretty' but this doesn't return any results:

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 0,
    "successful" : 0,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : 0.0,
    "hits" : [ ]
  }
}

I'm using:

• rsyslogd 8.2208.0 (aka 2022.08)
• logstash 7.17.8
• elastisearch 7.17.8

How can I solve this ?

Answer 1

Rsyslog is configured to send logs over TCP to logstash. Logstash is configured to listen for UDP messages.

To send logs over UDP, edit /etc/rsyslog.d/60-output.conf:

*.* @@localhost:10514;json-template        # note the second @ sign

To make it more clear, however, the RainerScript syntax can be used, which would look like the following:

# load omfwd module
module(load="omfwd")

*.* action(type="omfwd" target="127.0.0.1" port="10514" protocol="udp")