HAProxy does not forward to backend nodes - shows 503 Service unavailable

Question

Hello fellow Overflowers,

I have 2 Nginx Webservers in my OpenStack Enviroment. I'm trying to set up load balancing with HAProxy right now. Ubuntu 18 is the OS on all servers.

Added the backend IP's to the default config. When I try connect to my LB via Browser I get:

"503 Service Unavailable"

What I know so far:

• Backends are available when I connect directly to them.
• I opened the correct ports in the OpenStack GUI
• I checked the HAProxy logs and found the following:

Oct 20 13:04:30 HA_Proxy haproxy[2361]: [ALERT] 293/130430 (2361) : Starting frontend haproxynode: cannot bind socket [91.250.78.208:80]
Oct 20 13:04:30 HA_Proxy haproxy[2361]: Proxy backendnodes started.
Oct 20 13:04:30 HA_Proxy haproxy[2361]: Proxy backendnodes started.
Oct 20 13:04:30 HA_Proxy haproxy[2361]: Proxy stats started.
Oct 20 13:04:30 HA_Proxy haproxy[2361]: Proxy stats started.
Oct 20 13:05:27 HA_Proxy haproxy[2399]: Proxy haproxynode started.
Oct 20 13:05:27 HA_Proxy haproxy[2399]: Proxy haproxynode started.
Oct 20 13:05:27 HA_Proxy haproxy[2399]: Proxy backendnodes started.
Oct 20 13:05:27 HA_Proxy haproxy[2399]: Proxy backendnodes started.
Oct 20 13:05:27 HA_Proxy haproxy[2399]: Proxy stats started.
Oct 20 13:05:27 HA_Proxy haproxy[2402]: Server backendnodes/node1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions acti$

I dont know what do to with the "cannot bind socket" message, maybe its something in the config:

global
    log /dev/log    local0
    log /dev/log    local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

    # Default SSL material locations
    ca-base /etc/ssl/certs
    crt-base /etc/ssl/private

    # Default ciphers to use on SSL-enabled listening sockets.
    # For more information, see ciphers(1SSL). This list is from:
    #  https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
    # An alternative list with additional directives can be obtained from
    #  https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
    ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
    ssl-default-bind-options no-sslv3

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

frontend haproxynode
    bind *:80
    mode http
    default_backend backendnodes

backend backendnodes
    balance roundrobin
    option forwardfor
    http-request set-header X-Forwarded-Port %[dst_port]
    http-request add-header X-Forwarded-Proto https if { ssl_fc }
    option httpchk HEAD / HTTP/1.1\r\nHost:localhost
    server node1 192.168.0.77:8080 check
    server node2 192.168.0.76:8080 check

listen stats
    bind :32700
    stats enable
    stats uri /
    stats hide-version
    stats auth someuser:password

Anybody know what else I can check to solve the issue?

Also note that i started my apprenticeship in August and have almost no experience with loadbalancing or webservers at all =(

Answer 1

If you're getting a cannot bind socket error message then try to run the below command

setsebool -P haproxy_connect_any=1

Or else kill the service which was running on the port you want to use and then restart the haproxy

$fuser -k <your_port>/tcp

$sudo systemctl restart haproxy