Search code examples
kubernetesazure-container-service

Azure Kubernetes nginx-Ingress: preserve client IP


I try to preserve the client IP with proxy protocol. Unfortunately it does not work.

Azure LB => nginx Ingress => Service

I end up with the Ingress Service Pod IP.

Ingress Controller Deployment:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-lb
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      # hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
      # however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
      # that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
      # like with kubeadm
      # hostNetwork: true
      terminationGracePeriodSeconds: 60
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.5
        name: nginx-ingress-controller
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
        - --configmap=default/nginx-ingress-controller

Ingress Controller Service:

apiVersion: v1
kind: Service
metadata:
  name: nginx-ingress
  namespace: kube-system
  annotations:
   service.beta.kubernetes.io/external-traffic: "OnlyLocal"
spec:
  type: LoadBalancer
  ports:
    - port: 80
      name: http
    - port: 443
      name: https
  selector:
    k8s-app: nginx-ingress-lb

nginx config map:

apiVersion: v1
metadata:
  name: nginx-ingress-controller
data:
  use-proxy-protocol: "true"
kind: ConfigMap

Solution

  • Got it to work.

    In Ingress Controller Deployment I changed the image to

    gcr.io/google_containers/nginx-ingress-controller:0.8.3
    

    and removed the configmap.

    I am using ingress to forward to a pod with a dotnet core api.

    Adding

      var options = new ForwardedHeadersOptions()
      {
        ForwardedHeaders = Microsoft.AspNetCore.HttpOverrides.ForwardedHeaders.All,
        RequireHeaderSymmetry = false,
        ForwardLimit = null
      };
    
    //add known proxy network(s) here
    options.KnownNetworks.Add(network)
    app.UseForwardedHeaders(options);
    

    to Startup did the trick