Istio - Grafana with SubPath "if you're seeing this grafana has failed to load its application files"

Question

I am using Istio with Grafana enabled in AKS and want to use a subPath like example.com/metrics/grafana. The provided istio documentation only explains how to use it without a subpath, but a subdomain. But this is not an option here.

According to this grafana tutorial I have to set

domain = example.com
root_url = %(protocol)s://%(domain)s:%(http_port)s/metrics/grafana/
serve_from_sub_path = true

so I have set them in the IstioOperator:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  profile: default
  values:
    [...]
    grafana:
      # Enable Grafana deployment for analytics and monitoring dashboards
      enabled: true
      env:
        GF_SERVER_ROOT_URL: "%(protocol)s://%(domain)s/metrics/grafana/"
        GF_SERVER_DOMAIN: "example.com"
        GF_SERVER_SERVE_FROM_SUB_PATH: "true"

Additionally I have set up a VirtualService

    - name: grafana-route
      match:
        - uri:
            prefix: /metrics/grafana/
      route:
        - destination:
            host: grafana.istio-system.svc.cluster.local
            port:
              number: 3000

But when I visit example.com/metrics/grafana I keep getting the message:

If you’re seeing this Grafana has failed to load its application files

This could be caused by your reverse proxy settings.

If you host grafana under subpath make sure your grafana.ini root_path setting includes subpath

If you have a local dev build make sure you build frontend using: npm run dev, npm run watch, or npm > > run build

Sometimes restarting grafana-server can help

Anyone knows what the problem might be?

Answer 1

serve_from_sub_path must be set to false.

    grafana:
      # Enable Grafana deployment for analytics and monitoring dashboards
      enabled: true
      env:
        GF_SERVER_ROOT_URL: "%(protocol)s://%(domain)s/metrics/grafana/"
        GF_SERVER_DOMAIN: "example.com"
        GF_SERVER_SERVE_FROM_SUB_PATH: "false"

Furthermore the uri has to be rewritten:

    - name: grafana-route
      match:
        - uri:
            exact: /management/grafana
        - uri:
            prefix: /management/grafana/
      rewrite:
        uri: /
      route:
        - destination:
            host: grafana.istio-system.svc.cluster.local
            port:
              number: 3000