Search code examples
azureazure-blob-storageazure-aksnginx-ingress

AKS Ingress - multiple ExternalName services


we are migrating application to AKS. Originally it was deployed on our server and there was a directory with images on the same machine with proxy set. So we have www.example.com for application and www.example.com/images pointing to this directory.

We have successfully deployed our application to AKS with nginx ingress for different environments and we have created blob storage with directory for each environment, both working well. Now we need to create the same URL for application and its images.

I have found out that it can be achieved using ExternalName service (https://www.elvinefendi.com/2018/08/08/ingress-nginx-proxypass-to-external-upstream.html), but the target path in URL cannot be set in ExternalName service and is set in ingress metadata annotations and we need different paths for each environment.

I have found out examples of different ExternalName services but all served on the base URL or one ExternalName service same for all environments. So my question is - is there any way we can set up environment so as we have one ingress and one blob storage for all environments? Or is it better to have one ingress for each env?

Blob storage structure:

.
|--example-dev  
|  |-- dev-image.png 
|--example-test
|  |-- test-image.png
|--example-prod
|  |--prod-image.png

ExternalName service:

kind: Service
apiVersion: v1
metadata:
  name: file-service
spec:
  type: ExternalName
  externalName: example.blob.core.windows.net
  ports:
    - port: 443

Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - example.northeurope.cloudapp.azure.com
      secretName: tls-secret
  rules:
    - host: example.northeurope.cloudapp.azure.com
      http:
        paths:

#### Here we need to specify the path inside blob container somehow, I guess ####
          - path: /dev/images(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: file-service
                port:
                  number: 443
          - path: /dev(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: dev-example-app
                port:
                  name: http
          - path: /test(/|$)(.*)
            pathType: Prefix
            backend:
              service:
                name: test-example-app
                port:
                  name: http

Solution

  • We have finally solved it by creating separate ingress for each environment:

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt-prod
        nginx.ingress.kubernetes.io/rewrite-target: "/example-test/$2"
        nginx.ingress.kubernetes.io/upstream-vhost: "example.blob.core.windows.net"
      name: test-storage-ingress
    spec:
      ingressClassName: nginx
      tls:
        - hosts:
            - example.myserver.com
          secretName: tls-secret
      rules:
        - host: example.myserver.com
          http:
            paths:
              - path: /test/images(/|$)(.*)
                pathType: Prefix
                backend:
                  service:
                    name: storage-service
                    port:
                      number: 80
    

    So we have one ingress for all webapps and one for images for each environment so as it does not get confusing.