Kubernetes AKS ingress & MVC routing

Question

I'm currently in the process of setting up a Kubernetes AKS cluster, mostly for learning purposes for myself and as a proof of concept. My goal is like this:

• CI / CD with Azure DevOps
  • Every commit to the master-branch triggers an automated release to Kubernetes in the namespace "production"
  • Every commit to a non-master-branch triggers an automated release to Kubernetes in the namespace "development"
• Kubernetes management
  • Resources are created via Helm
  • Azure DevOps task is used to deploy these resources in combination with variable-groups per environment
• Application
  • Simple MVC app
  • Real database outside of Kubernetes, Azure SQL Server

Now, everything works smoothly, but I really struggle with the logical concept of "namespaces as environments". I know, this isn't a great idea in the first place, as for example, the development-environment could use up all the resources. But as I'm hosting stuff myself, I didn't want to create multiple clusters and I think for starters, having namespaces as environments is reasonable. My problem comes with the routing: my pretty naive approach is explained here Kubernetes Cross Namespace Ingress Network:

• One ingress in the default namespace
• Per environment, one ExternalName-Service in the default namespace with a suffix, targetting the CNAME of the service in the other namespaces
• Per environment namespace, one ClusterIP service

The ingress looks like this:

  rules:
    - host: 09ab799fd5674c4594a5.centralus.aksapp.io
      http:
        paths:
          - path: /dev
            pathType: Prefix
            backend:
              service:
                name: {{include "helmakskubernetespoc-chart.fullname" .}}external{{ .Values.namespaces.development }}
                port:
                  number: {{ .Values.externalService.port }}
          - path: /
            pathType: Prefix
            backend:
              service:
                name: {{ include "helmakskubernetespoc-chart.fullname" .}}external{{ .Values.namespaces.production }}
                port:
                  number: {{ .Values.externalService.port }}

My goal being, that no subroute targets production and the subroute /dev targeting development.

This works as long as there isn't a subroute aka for production. But as I've learned here Kubernetes Ingress non-root path 404 Not Found, the ingress takes the route and passes it 1:1 to the internal service. As my little MVC app doesn't know "/dev", I get a 404. So far so good, I can change this via annotation:

nginx.ingress.kubernetes.io/rewrite-target: /

Now, every call to /dev does in fact route to the correct app, BUT every click on a button routes back to production, as the initially added /dev is lost. Also, trying this one Ingress don't load the website assets (Css files and Javascript files), I have the same problem.

I'm wondering if I'm conceptually making a mistake: Ingress rules are, as far as I know, only working with subroutes, therefore I can't for example use a port. Is possibly ingress the wrong way to split the environments? As the MVC app either never receives the /dev or doesn't know what to do with it, I don't see an elegant solution for this problem?

Answer 1

You probably wanna use different hosts per environment. Something like:

rules:
    - host: dev.09ab799fd5674c4594a5.centralus.aksapp.io
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: {{include "helmakskubernetespoc-chart.fullname" .}}external{{ .Values.namespaces.development }}
                port:
                  number: {{ .Values.externalService.port }}
    - host: 09ab799fd5674c4594a5.centralus.aksapp.io
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: {{ include "helmakskubernetespoc-chart.fullname" .}}external{{ .Values.namespaces.production }}
                port:
                  number: {{ .Values.externalService.port }}