Search code examples
kubernetestraefiktraefik-ingressk3s

Making a sub-path the 'root' path of my backend services on k3s?


I have a k3s cluster with a Prometheus and Alertmanager deployment inside, and set up an Ingress resource for each of them. My initial setup was prometheus.domain.com and alermanager.domain.com respectively, and these worked as expected.

However, I would like to switch it to domain.com/prometheus and domain.com/alertmanager respectively, and the options I have researched aren't working.

GOAL Have my Prometheus service work having domain.com/prometheus being the 'root' of my Prometheus pathing, hence redirecting automatically to domain.com/prometheus/graph, as per its default behavior, and all subpaths under Prometheus (alerts, config, admin API) follow the same behavior.

Attempt #1

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  app: Prometheus
  annotations:
    traefik.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
    - http:
        paths:
          - path: /prometheus($|/)(.*)
            backend:
              serviceName: prometheus
              servicePort: 9090

SSH into the Vagrant box hosting my k3s cluster:

$ kubectl get ingress
NAME         CLASS             HOSTS         ADDRESS         PORTS
prometheus   <none>            *             192.168.0.200   80

$ curl 192.168.0.200/prometheus
404 page not found

$ curl 192.168.0.200/prometheus/graph
404 page not found

Attempt #2

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  app: Prometheus
  annotations:
    traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
spec:
  rules:
    - http:
        paths:
          - path: /prometheus
            backend:
              serviceName: prometheus
              servicePort: 9090

Same result as above.


Solution

  • You need to start prometheus with:

    --web.route-prefix="http://example.com/prometheus"
    

    From source code:

    web.external-url", "The URL under which Prometheus is externally reachable (for example, if Prometheus is served via a reverse proxy). Used for generating relative and absolute links back to Prometheus itself. If the URL has a path portion, it will be used to prefix all HTTP endpoints served by Prometheus. If omitted, relevant URL components will be derived automatically.


    And for altermanager you need to set:

    --alertmanager.url=http://example.com/alertmanager
    

    NOTE: Don't use rewrites in ingress in this case.