I'm currently attempting to use the logback-logstash-encoder to write my logs to two different logstash instances. Both of these instances will be writing to the same Elasticsearch instance.
I'm struggling to find a way to load balance between the two logstash instances.
After reading both the logback documentation and the log4j2 documentation, its clear that the TcpAppender that logback-logstash uses does not support 'load-balanced' urls (i.e. url1, url2). In log4j2, I can approximate this behavior with the FailoverAppender.
Is there similar functionality in logback? Or will I need to stand up another service to load-balance for logback?
AFAIK there is no support for load balancing within TcpAppenders. However, you could achieve load balancing by adding a hardware/software load balancer in front of the logstash boxes. In case a logstash process/box dies, the TCP connection gets reset, and the next log event should cause to re-establish the connection again.
This approach would require adding a load balancer to your setup. You would use the load balancer's address in your TcpAppender. The balancer handles the availability of logstash for you.
It depends on what you want to achieve. Is it not losing any log events? Then perhaps the better approach would be to write the logs into files and push the files by logstash-forwarder to your logstash boxes. If you're concerned about connection timeouts and slowdown of your application, then either async-appender or using UDP could be your choice.
HTH, Mark