Search code examples
kubernetes-ingressnginx-ingressactivemq-artemis

Configure Master/Slave routing in Ingress Nginx


I want a master/slave setup for Artemis ActiveMQ on K8 cluster using stateful set. So here is what I am doing. Creating two stateful set with service for Master and Slave. The service looks something like this

NAME                TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                          AGE
c1-artemis-master-svc    LoadBalancer   10.110.0.76      127.0.0.1     8161:30011/TCP,61616:31211/TCP   10s
c1-artemis-slave-svc   LoadBalancer   10.111.165.162   127.0.0.1     8261:30064/TCP,62616:30649/TCP   17s

Now using ingress nginx to Expose the TCP port 61616 and 62616 https://mailazy.com/blog/exposing-tcp-udp-services-ingress/

So, I have configured the values.yaml in ingress as

tcp:
  61625: "default/c1-artemis-master-svc:61616"
  61626: "default/c1-artemis-slave-svc:62616"

I can use 61625 for master and 61626 for slave.

But I want one port which is exposed from nginx and that port forwards the request either to 61625 or 61626 accroding to the availibilty.

For example, I want to conenct to port 9100 which will forward the request to either master(if it is running) or slave.

Is there a way I can configure some routing mechanism inside ingress nginx.

When using standalone mster/slave setup (not in k8 cluster but running just the docker images, one for master, one for slave), i had setup HAProxy for auto routing something like this -

defaults
  mode tcp
  timeout client 10s
  timeout connect 5s
  timeout server 10s
  timeout http-request 10s
  timeout queue           1m
  timeout connect         10s
  timeout client          1m
  timeout server          1m
  timeout http-keep-alive 10s
  timeout check           10s
  maxconn                 3000

frontend myfrontend
  mode tcp
  bind 0.0.0.0:9100
  default_backend pairnode

backend pairnode
  mode tcp
  # check for master node
  server ndoe1 0.0.0.0:61616 check
  # check for slave node
  server ndoe2 0.0.0.0:62616 check backup

I want something similar in ingress nginx.


Solution

  • I used readiness probe to check if the TCP connection of broker is up and running, and routed traffic accordingly.

    readinessProbe:
      tcpSocket:
        port: 61616
      initialDelaySeconds: 5
      periodSeconds: 10