Logstash: Feed whole file and create a new event by splitting with newline character

Question

I have the following logstash configuration for reading syslog-like messages from kafka:

input {
    kafka {
        bootstrap_servers => "172.24.0.3:9092"
        topics => ["test"]
    }
}
filter {
    grok {
        match => { "message" => "%{SYSLOGTIMESTAMP}" }
    }
}
output {
    stdout { codec => rubydebug }
}

So, when a syslog-line is sent at logstash input the following message is generated at stdout:

FROM KAFKA

r = p1.send('test', b'Jul 16 09:07:47 ubuntu user: test500')

STDOUT

{
       "message" => "Jul 16 09:07:47 ubuntu user: test500",
      "@version" => "1",
    "@timestamp" => 2018-07-16T12:29:57.854Z,
          "host" => "6d87dde4c74e"
}

Now, I would like to add multiple lines with \n character at the end of each line and logstash processes the input as two separated messages so that the logstash stdout to be similar to the following example:

MULTIPLE LINES FROM KAFKA IN THE SAME MESSAGE

r = p1.send('test', b'Jul 16 09:07:47 ubuntu user: test501\nJul 16 09:07:47 ubuntu user: test502')

DESIRED STDOUT

{
       "message" => "Jul 16 09:07:47 ubuntu user: test501",
      "@version" => "1",
    "@timestamp" => 2018-07-16T12:29:57.854Z,
          "host" => "6d87dde4c74e"
}
{
       "message" => "Jul 16 09:07:47 ubuntu user: test502",
      "@version" => "1",
    "@timestamp" => 2018-07-16T12:29:57.854Z,
          "host" => "6d87dde4c74e"
}

Any ideas how to achieve this behavior on logstash?

Answer 1

I managed to achieve the behavior I described above by using line codec:

input {
    kafka {
        bootstrap_servers => "172.24.0.3:9092"
        topics => ["test"]
        ## ## ## ## ## 
        codec => line
        ## ## ## ## ##
    }
    stdin {}
}