AKS can't modify AGIC on ingress creation due to the policy

Question

I've just finished setting up AKS with AGIC and using Azure CNI. I'm trying to deploy NGINX to test if I set the AKS up correctly with the following configuration:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  annotations:
    kubernetes.io/ingress.class: azure/application-gateway
    kubernetes.io/ingress.allow-http: "false"
    appgw.ingress.kubernetes.io/use-private-ip: "false" 
    appgw.ingress.kubernetes.io/override-frontend-port: "443"
spec:
  tls:
    - hosts:
        - my.domain.com
      secretName: aks-ingress-tls
  rules:
    - host: my.domain.com
      http:
          paths:
          - path: /
            pathType: Prefix 
            backend:
              service:
                name: nginx-service
                port:
                  number: 80

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      component: nginx
  template:
    metadata:
      labels:
        component: nginx
    spec:
      containers:
        - name: nginx
          image: nginx
          ports:
            - containerPort: 80

apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: ClusterIP
  selector:
    component: nginx
  ports:
    - port: 80
      protocol: TCP

There's no error or any other log message on apply the above configurations.

> k apply -f nginx-test.yml
deployment.apps/nginx-deployment created
service/nginx-service created
ingress.networking.k8s.io/nginx-ingress created

---

But after a further investigation in the Application Gateway I found these entries in the Activity log popped up at the same time I applied the said configuration.

Further details in one of the entries is as follows:

• Operation name: Create or Update Application Gateway
• Error code: RequestDisallowedByPolicy
• Message: Resource 'my-application-gateway' was disallowed by policy.

  [
    {
      "policyAssignment": {
        "name": "Encryption In Transit",
        "id": "/providers/Microsoft.Management/managementGroups/***/providers/Microsoft.Authorization/policyAssignments/EncryptionInTransit"
      },
      "policyDefinition": {
        "name": "HTTPS protocol only on Application Gateway listeners",
        "id": "/providers/microsoft.management/managementgroups/***/providers/Microsoft.Authorization/policyDefinitions/HttpsOnly_App_Gateways"
      },
      "policySetDefinition": {
        "name": "Encryption In Transit",
        "id": "/providers/Microsoft.Management/managementgroups/***/providers/Microsoft.Authorization/policySetDefinitions/EncryptionInTransit"
      }
    }
  ]
  

My organization have a policy to enforce TLS but from my configuration I'm not sure what I did wrong as I have already configured the ingress to only use HTTPS and also have certificate (from the secret) installed.

I'm not sure where to look and wish someone could guide me in the correct direction. Thanks!

Answer 1

• As you said, your organization has a policy for enforcing TLS for securing encrypted communication over HTTPS. Therefore, when you create an ‘NGINX’ deployment through the ‘yaml’ file posted, you can see that the nginx application is trying to connect to the application gateway ingress controller over Port 80 which is reserved for HTTP communications. Thus, your nginx application has also disallowed the usage of private IPs with the AGIC due to which the nginx application is directly overriding the HTTPS 443 port for reaching out to the domain ‘my.domain.com’ over port 80 without using the SSL/TLS certificate-based port for communication.

Thus, would suggest you to please configure NGINX application for port 443 as the frontend port for the cluster IP and ensure ‘SSL redirection’ is set to enabled due to which when the NGINX application is deployed, it will be not face the policy restrictions and get failed. Also, refer to the below snapshot of the listeners in application gateway and load balancer when provisioning an AGIC for an AKS cluster.

Also, for more detailed information on deploying the NGINX application in AKS cluster on ports, kindly refer to the below documentation link: -

https://learn.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli