websocketsignalrhaproxy
HAProxy and websockets never connect
I thought with HAProxy 2.4 supporting WebSockets over h2, I would finally be able to get websockets over HAProxy to work... But no...
When my browser tries to connect, it instantly gives up on the "connect?transport=webSockets" and tries SSE. This "connect" also is instantly abandoned and yet it "start?transport=SSE" just fine.
Logging in HAProxy says it never even got the requests for connect.
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:18.520] www-httpsStrict~ test_backend/werdc2020 557/0/0/2/558 200 704 - - --VN 2/1/0/0/0 0/0 "GET https://myserver.com/signalr/negotiate?clientProtocol=2.1&connectionData=%5B%7B%22name%22%3A%22testservice%22%7D%5D&_=1637143818090 HTTP/2.0"
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:19.075] www-httpsStrict~ test_backend/werdc2020 52/0/0/1/52 404 1392 - - --VN 2/1/0/0/0 0/0 "GET https://myserver.com/favicon.ico HTTP/2.0"
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:19.279] www-httpsStrict~ test_backend/werdc2020 96/0/0/7/103 200 339 - - --VN 3/2/1/1/0 0/0 "GET https://myserver.com/signalr/start?transport=serverSentEvents&clientProtocol=2.1&connectionToken=%2FZ99uK3N88wIyXYxQlQc4w42M3jcf0Tz6tbLcm7CXhOcTgTu7qgbfsMxn9l6GNP%2FML39o%2BabOZ4GSQDgAxo7oYXIacee8Ku2Nd1QXvWRalisPY%2BTIgddfJsWdE828LaH&connectionData=%5B%7B%22name%22%3A%22testservice%22%7D%5D&_=1637143818091 HTTP/2.0"
Nov 17 17:10:19 127.0.0.1 haproxy[26334]: 192.168.126.78:1228 [17/Nov/2021:17:10:19.375] www-httpsStrict~ test_backend/werdc2020 65/0/0/20/84 200 348 - - --VN 3/2/1/1/0 0/0 "POST https://myserver.com/signalr/send?transport=serverSentEvents&clientProtocol=2.1&connectionToken=%2FZ99uK3N88wIyXYxQlQc4w42M3jcf0Tz6tbLcm7CXhOcTgTu7qgbfsMxn9l6GNP%2FML39o%2BabOZ4GSQDgAxo7oYXIacee8Ku2Nd1QXvWRalisPY%2BTIgddfJsWdE828LaH&connectionData=%5B%7B%22name%22%3A%22testservice%22%7D%5D HTTP/2.0"
If I connect directly on the server it works just fine...
HAProxy config:
global
log 127.0.0.1 local0 debug
log-tag haproxy
maxconn 10000
user haproxy
group haproxy
daemon
nbproc 1
nbthread 8
#cpu-map auto:all 0-1
stats socket /var/run/haproxy.sock mode 600 level admin
stats timeout 30s
pidfile /var/run/haproxy.pid
tune.ssl.default-dh-param 2048
tune.ssl.cachesize 100000
tune.ssl.lifetime 600
tune.ssl.maxrecord 1460
# intermediate configuration
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:!MD5:!aNULL:!DH:!RC4
ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5s
timeout client 1m
timeout client-fin 1m
timeout server 1h
timeout tunnel 1h
frontend stats
bind 0.0.0.0:8080
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth myuser:1111
stats refresh 60s
# Force front server MUST use redirect on HTTPS
frontend www-httpStrict
bind *:80
option splice-auto
# Test URI to see if its a letsencrypt request
acl letsencrypt-acl path_beg /.well-known
acl test hdr_sub(cookie) istest=true
redirect scheme https code 307 if !{ ssl_fc } !letsencrypt-acl
use_backend letsencrypt if letsencrypt-acl
use_backend test_backend if test
default_backend www-backendStrict
frontend www-httpsStrict
bind *:443 tfo ssl crt /etc/ssl/private alpn h2,http/1.1
option forwardfor
option splice-auto
acl test hdr_sub(cookie) istest=true
use_backend test_backend if test
default_backend www-backendStrict
backend letsencrypt
server nginx 127.0.0.1:8888
# this backend require haproxy open SSL tunel to port 443 on webservers
backend www-backendStrict
balance roundrobin
cookie MyWebFarm insert
option forwardfor
option splice-auto
option tcp-smart-connect
option httpchk
http-check connect ssl alpn h2 sni myserver.com
http-check send meth HEAD uri /login.aspx ver HTTP/2 hdr Host myserver.com
http-check expect status 200-399
http-request add-header X-Forwarded-Proto https if { ssl_fc }
retry-on all-retryable-errors
http-request disable-l7-retry if METH_POST
default-server ssl tfo verify none alpn h2,http/1.1 check allow-0rtt
server werdc01 192.168.1.5:443 cookie werc01 check
server werdc2020 192.168.1.6:443 cookie werdc2020 check
backend test_backend
balance roundrobin
cookie MyWebFarm insert
option forwardfor
option splice-auto
option tcp-smart-connect
option httpchk
http-check connect ssl alpn h2 sni myserver.com
http-check send meth HEAD uri /login.aspx ver HTTP/2 hdr Host myserver.com
http-check expect status 200-399
http-request add-header X-Forwarded-Proto https if { ssl_fc }
retry-on all-retryable-errors
http-request disable-l7-retry if METH_POST
default-server ssl verify none alpn h2,http/1.1 check allow-0rtt
server werdc2020 192.168.1.6:443 cookie werdc2020 check
I have the istest cookie set to always hit the test_backend...
Headers sent from browser:
GET /signalr/connect?transport=webSockets&clientProtocol=2.1&connectionToken=%2FZ99uK3N88wIyXYxQlQc4w42M3jcf0Tz6tbLcm7CXhOcTgTu7qgbfsMxn9l6GNP%2FML39o%2BabOZ4GSQDgAxo7oYXIacee8Ku2Nd1QXvWRalisPY%2BTIgddfJsWdE828LaH&connectionData=%5B%7B%22name%22%3A%22catiservice%22%7D%5D&tid=7 undefined
Host: myserver.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:94.0) Gecko/20100101 Firefox/94.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Sec-WebSocket-Version: 13
Origin: https://myserver.com
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: P1GaqaLxIxBfuXDCtxuQwg==
DNT: 1
Connection: keep-alive, Upgrade
Cookie: _ga=GA1.2.95033637.1627965955; .MyAuth=D758016C45CF1B9BDC38D17DCAF7A3CEE7E528B97352267A1B0D019BFE20964BF2900C655950E126CD7FC08CB421C1A997D1A5FDB0266316B920634FB2C23B63080B271B6331096902B47BF73661240D; MyWebFarm=werdc2020; ASP.NET_SessionId=w1zibnnuht4ohicpntyhn55q; istest=true
Sec-Fetch-Dest: websocket
Sec-Fetch-Mode: websocket
Sec-Fetch-Site: same-origin
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
TE: trailers
Why doesn't HAProxy see the "connect" requests?
Why doesn't it connect with WebSockets?
Solution
There is a github issue on this: https://github.com/haproxy/haproxy/issues/162
Although your question isn't very insightful, I'd recommend to strip down your config to the bare minimum to test websocket.
Here's a snippet to consider
option http-server-close
option http-use-htx
acl hdr_connection_upgrade hdr(Connection) -i upgrade
acl hdr_upgrade_websocket hdr(Upgrade) -i websocket
Why doesn't HAProxy see the "connect" requests?
I suppose, just because the CONNECT has not been successful. If you'd use tcpdump you pretty sure would see a request incoming.