Search code examples
configreverse-proxyhaproxy

reverse proxy setup using subdirs using haproxy

I have trouble getting my reverse proxy setup to work using haproxy 1.6.3 running on ubuntu 16. Heres what i am trying to achieve:

  • on my main machine, i have a webapplication with apache running under https://bar.com that uses mod_rewrite for routing
  • various internal servers (machine 1..n) are linked via vpn to the main machine, all exposing only a http interface with websocket support on port 8081 of the vpn adapter
  • I want the internal machines to be accessible via a subdirctory of the main machine, eg. for machine 1 I would like to access its webpages via https://bar.com/machine1 - with the internal traffic being http, and the traffic from the main machine to the visitor being secured by the ssl certificate of the main machine
  • All other traffic that does not match /machine1 should not be affected and should be served by main machines apache as before
  • Paths on the forwarded webinterface of the machines 1..n are not an issue, as they are able to dynamically modify their paths via a header directive (Orig-Path and X-Script-Path) in the below config
  • apache2 on the main machine is configured to only listen on localhost with Listen 127.0.0.1 and all traffic to the main site is handled by haproxy via the default backend

Heres the relevant configs:

  • apache .htaccess of main site (excluding the relevant subdirectory)

    # Exclude machine1 subdirectory from rewrite
RewriteRule ^(machine1)($|/) - [L]
RewriteCond %{REQUEST_URI} !^/index\.php
RewriteCond %{SCRIPT_FILENAME} !-f
RewriteCond %{SCRIPT_FILENAME} !-d
RewriteRule .* index.php [L]

  • haproxy setup

    frontend http-in
    bind <external-ip>:80
    mode tcp
    option tcplog
    acl machine1 path_beg /machine1
    use_backend machine1-backend if machine1
    default_backend default-backend-http

frontend https-in
    bind <external-ip>:80
    mode tcp
    option tcplog
    default_backend default-backend-https

backend machine1-backend
    reqrep ^([^\ :]*)\ /machine1/(.*)  \1\ /\2
    http-request set-header Orig-Path /machine1/
    http-request set-header X-Script-Path /machine1/
    http-request set-header Host bar.com
    option http-server-close  
    server m1 10.0.0.4:8081

backend default-backend-https
    server main 127.0.0.1:443
    mode tcp

backend default-backend-http
    server main 127.0.0.1
    mode tcp

The issues/questions i currently have:

  • accessing the unencrypted variant (http://bar.com/machine1) sometimes serves the right page of machine1, but most of the time, i get a 404 from the main machines apache - i thought this could be solved with option http-server-close but it's not - could someone point me towards what i am missing here ? I verified in the rare cases of a valid response that paths are correctly extended with /machine1 - e.g. /machine1/css/main.css for css includes - but even after a scccessful initial pull from machine1 - all subsequent fetches for scripts, images and css return a 404 again

  • i wasn't able to figure out how to properly setup ssl to use on the /machine1 requests with haproxy handling https->http traffic conversion, so the ssl part does not include the routing at the moment - how do i need to extend the config to make this work for https://bar.com/machine1? (Assuming valid certificate for bar.com existing under /etc/keys/web.pem)

Bonus questions:

  • Is there any chance to make this configuration dynamic ? e.g. pull the relevant info (server ip, subdirectory name) from a database / use some sort of logic here, as the machine 1...n links will change during runtime (new machines connect, others disconnect, and the number of possible machines is quite large)
  • Is there an option to validate users before using /machine1 ? Main web application does user management, so i would preferably check if a user is authenticated before allowing access to /machine1 - can this be done ?
Solution

  • Answering my own question here: After some research, issue was wrong mode tcp in config for this use case, and is easily solved by switching mode in front and backend to http. From the docs

    • mode tcp:

    In this mode, HAProxy doesn’t decipher the traffic. It just opens a TCP tunnel between the client and the server and let them together negotiate and handle the TLS traffic.

    When using this mode, HAProxy does not evaluate the HTTP headers in the packet. In this case one obivously doesn't have the option to differentiate backends on http specific headers like uri, thats why the initial configuration does not work.

    • mode http:

    In this mode, HAProxy decipher the traffic on the client side and re-encrypt it on the server side. It can access to the content of the request and the response and perform advanced processing over the traffic.

    In this case, all http header fields are available to haproxy for backend selection.

    This of course has implications on ssl - there are various variants for this setup, i have chosen to use SSL/TLS offloading and let HAProxy decipher the traffic on the client side and connect in clear to the internal servers.

    That makes the machine running haproxy the sslendpoint, and ssl certificates need to be set up here instead of on the webserver. Additionally, with this setup, the webservers running the webapp can be completely isolated, only serving pages internally to the haproxy machine. This also answers question 2.

    And lastly, for the bonus questions:

    • I've gone the route to imlement 'dynamic' configuration via shell scripts that modify haproxy config on the fly when new machines connect with infos from the database, bringing the changes live with service haproxy reload (ubuntu) - this seems to work very well.
    • For user authentication, i have it set up that the backend machines now query the main webapplication for authorization and redirect to main webapplication if authorization is denied. For validation, a cookie (or the absence of it) is used. I'm currently testing this setup, but right now it looks like this will work.

    And finally, the resulting (working) configuration i have (please note, i've added http to https redirection on the haproxy as well):

    frontend http-in
    bind <external-ip>:80
    bind <external-ip>:443 ssl crt /path/to/cert/cert.pem
    acl machine1 path_beg /machine1
    reqadd X-Forwarded-Proto:\ https
    mode http
    option httplog
    use_backend machine1-backend if machine1
    default_backend default-backend

backend default-backend
    redirect scheme https if !{ ssl_fc }
    server main 127.0.0.1:80
    mode http

backend machine1-backend
    http-request set-header Orig-Path /machine1/
    http-request set-header X-Script-Path /machine1/
    http-request set-header Host bar.com
    reqirep ^([^\ :]*)\ /machine1/(.*)  \1\ /\2
    server m1 10.0.0.4:8081