Following is the output snippet from my filebeat.yml
output:
logstash:
hosts: ['192.168.200.38:5015']
where 192.168.200.38:5015 is a haproxy server listening in tcp mode.
Following is my haproxy configuration:
global
daemon
maxconn 256
defaults
mode tcp
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
timeout tunnel 1h
listen stats
bind 0.0.0.0:9999
stats enable
stats hide-version
stats uri /stats
frontend proxy_in
bind 0.0.0.0:5015
backend proxies_out
balance roundrobin
mode tcp
server ip-1 listener.logz.io:5015
Using the above configuration I get the following error:
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.560Z DEBUG [logstash] logstash/async.go:111 connect
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.562Z INFO pipeline/output.go:105 Connection to backoff(async(tcp://192.168.200.38:5015)) established
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.563Z DEBUG [logstash] logstash/async.go:159 1 events out of 1 events sent to logstash host 192.168.200.38:5015. Continue sending
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.564Z DEBUG [transport] transport/client.go:218 handle error: EOF
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.564Z DEBUG [transport] transport/client.go:131 closing
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.564Z ERROR logstash/async.go:256 Failed to publish events caused by: EOF
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.565Z DEBUG [logstash] logstash/async.go:159 1 events out of 1 events sent to logstash host 192.168.200.38:5015. Continue sending
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.565Z DEBUG [logstash] logstash/async.go:116 close connection
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.565Z ERROR logstash/async.go:256 Failed to publish events caused by: client is not connected
Jul 02 14:32:46 cust1-bast-linux-0 filebeat[83565]: 2019-07-02T14:32:46.565Z DEBUG [logstash] logstash/async.go:116 close connection
I am using haproxy server as a proxy server here. Earlier I had squid proxy(http) which will not work with filebeat. So changed it to haproxy and I have listener.logz.io:5015 as the backend.
What am I doing wrong here?
Solved!
The problem was 'listener.logz.io:5015' expects a certificate. Added ssl option to pass certificate, but in order for haproxy to identify the certificate, I had to change the hostname to something.logz.io. So I had changed the hostname where haproxy was installed to proxy.logz.io(I know its a workaround).
The following configurations worked:
filebeat.yml
output:
logstash:
hosts: ['proxy.logz.io:5015']
ssl:
certificate_authorities: ['/etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt']
haproxy.cfg
#Forward HAProxy Config
global
debug
daemon
maxconn 10000
defaults
mode tcp
timeout client 200000ms
timeout server 200000ms
#create new frontend to process 5015
frontend https_frontend
bind *:5015
mode tcp
option tcplog
default_backend logz
#Define backend for above frontend
backend logz
mode tcp
balance roundrobin
server logzio listener.logz.io:5015