I have a simple and straightforward config and I'm not sure what I'm doing wrong trying to get this multiline working.
input {
gelf {
codec => multiline {
pattern => "^%{TIMESTAMP_ISO8601} "
negate => true
what => "previous"
}
}
}
filter {}
output {
# I have the relevant ES hosts & index here
elasticsearch { }
stdout {
codec => rubydebug
}
}
And I'm testing it as below and I get single lines, multiline tag doesn't get added and I see it in logstash debug logs that events go through filter one by one.
docker run -it --log-driver gelf --log-opt gelf-address=udp://127.0.0.1:12201 \
--log-opt tag=mline-test python:alpine \
python -c 'print("[2017-10-18 00:00:00,000] Hello world");assert False'
I have discovered that the codecs are ignored in gelf input plugin: logstash-plugins/logstash-input-gelf#37
This means that the only multiline option with gelf is using logstash-2.4 and taking the single-thread performance hit.
Settings: Default pipeline workers: 4
Defaulting pipeline worker threads to 1 because there are some filters that might not work with multiple worker threads {:count_was=>4, :filters=>["multiline"], :level=>:warn}
Pipeline main started