Search code examples
dockerartifactoryhaproxy

Configuring haproxy load balancer in front of ha artifactory cluster

I'm trying to configure an haproxy load balancer in front of our 2-node ha artifactory cluster. I'm using the page here as a guide:

https://jfrog.com/knowledge-base/how-to-configure-haproxy-with-artifactory/

but this was written years ago for a much older version of haproxy (I'm running 2.0.8) and a lot of the code is deprecated. The recommended configuration starts with errors. Here it is:

# version 1.0
# History
# https://jfrog.com/knowledge-base/how-to-configure-haproxy-with-artifactory/
# —————————————————————————
# Features enabled by this configuration
# HA configuration
# port 80, 443  Artifactory GUI/API
#
# This uses ports to distinguish artifactory docker repositories
# port 443  docker-virtual (v2) docker v1 is redirected to docker-dev-local.
# port 5001 docker-prod-local (v1); docker-prod-local2 (v2)
# port 5002 docker-dev-local (v1); docker-dev-local2 (v2)
#
# Edit this file with required information enclosed in <…>
# 1. certificate and key
# 2. artifactory-host
# 3  replace the port numbers if needed
# —————————————————————————-
global
    log 127.0.0.1   local0
    chroot /var/lib/haproxy
    maxconn 4096
    user haproxy
    group haproxy
    daemon
    tune.ssl.default-dh-param 2048
    stats socket /run/haproxy/admin.sock mode 660 level admin

defaults
    log global
    mode http
    option  httplog
    option  dontlognull
    option  redispatch
    option  forwardfor
    option  http-server-close
    maxconn 4000
    timeout connect 5000
    timeout client 50000
    timeout server 50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

listen stats
  bind *:2016
  mode http
  stats enable
  stats uri /haproxy
  stats hide-version
  stats refresh 5s
  stats realm Haproxy\ Statistics

frontend normal
    bind *:80
    bind *:443 ssl crt /etc/ssl/artifactory/cert.pem
    mode http
    option forwardfor
    reqirep ^([^ :]*) /v2(.*$) 1 /artifactory/api/docker/docker-virtual/v22
    reqadd X-Forwarded-Proto: https if { ssl_fc }
    option forwardfor header X-Real-IP
    default_backend normal

# Artifactory HA Configuration
# Using default failover interval – rise = 2; fall =3 3; interval – 2 seconds
backend normal
    mode http
    balance roundrobin
    option httpchk OPTIONS /
    option httpchk GET /api/system/ping HTTP/1.1\r\nHost:haproxy\r\n
    option forwardfor
    option http-server-close
    appsession JSESSIONID len 52 timeout 3h
    server platform-artifactory-ha-01 172.17.1.71:80 check fall 3 inter 3s rise 2
    server platform-artifactory-ha-02 172.17.1.122:80 check fall 3 inter 3s rise 2

If I run haproxy -f haproxy.cfg -c I get:

[WARNING] 121/054551 (11113) : parsing [haproxy.cfg:55] : The 'reqirep' directive is deprecated in favor of 'http-request replace-header' and will be removed in next version.
[ALERT] 121/054551 (11113) : parsing [haproxy.cfg:55] : 'reqirep' : Expecting nothing, 'if', or 'unless', got '/v2(.*$)'.
[WARNING] 121/054551 (11113) : parsing [haproxy.cfg:56] : The 'reqadd' directive is deprecated in favor of 'http-request add-header' and will be removed in next version.
[ALERT] 121/054551 (11113) : parsing [haproxy.cfg:56] : 'reqadd' : Expecting nothing, 'if', or 'unless', got 'https'.
[ALERT] 121/054551 (11113) : parsing [haproxy.cfg:68] : 'appsession' is not supported anymore since HAProxy 1.6.
[ALERT] 121/054551 (11113) : Error(s) found in configuration file : haproxy.cfg
[ALERT] 121/054551 (11113) : Fatal errors found in configuration.

I've been able to get artifactory to start up by commenting the following lines 64 and 65:

    #    reqirep ^([^ :]*) /v2(.*$) 1 /artifactory/api/docker/docker-virtual/v22
    #    reqadd X-Forwarded-Proto: https if { ssl_fc }

and adding:

http-request set-header X-Forwarded-Proto https if { ssl_fc }

to replace line 65

I also had to comment line 79 to get the haproxy service to start without errors:

   # appsession JSESSIONID len 52 timeout 3h

But now it doesn't properly work in the case where folks are trying to push dockers into the regsitry.

I've got to figure the new way to write line 79 and line 64. But I'm having trouble finding the correct configuration directives in the documentation.

Solution

  • The reqirep keyword was spitted in several http-request directives.
    You will need to use http-request replace-path.

    My suggestion, untested

    # reqirep ^([^ :]*) /v2(.*$) 1 /artifactory/api/docker/docker-virtual/v22
http-request replace-path /v2(.*$) /artifactory/api/docker/docker-virtual/v22\1

    The appsession isn't anymore part of haproxy as the ALERT message shows.

    My suggestion for the cookie sticky, untested.

    backend normal
  mode http
  balance roundrobin
  # this makes no sense option httpchk OPTIONS /
  option httpchk GET /api/system/ping HTTP/1.1\r\nHost:haproxy\r\n
  option forwardfor
  option http-server-close

  stick-table type string len 52 size 2m expire 3h

  #appsession JSESSIONID len 52 timeout 3h
  stick on cookie(JSESSIONID) 

  server platform-artifactory-ha-01 172.17.1.71:80 check fall 3 inter 3s rise 2
  server platform-artifactory-ha-02 172.17.1.122:80 check fall 3 inter 3s rise 2