Network Policy for GCP Cloud MySQL Proxy

Question

I am trying to write some Network Policies for my App, but the database connection fails as soon as I add my policy.

Here is said that the MySQL Proxy uses Ports TCP:3307 and 443 https://cloud.google.com/sql/docs/mysql/sql-proxy#how-works

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: {{ template "name" . }}
spec:
  podSelector:
    matchLabels:
      app: {{ template "name" . }}
  policyTypes:
  - Egress
  egress:
  # allow DNS resolution
  - ports:
    - port: 53
      protocol: UDP
    - port: 53
      protocol: TCP
    - port: 443
      protocol: TCP
    - port: 3307
      protocol: TCP

EDIT: Deployment snippet:

  - name: cloudsql-proxy
    image: gcr.io/cloudsql-docker/gce-proxy:1.28.0
    command: ["/cloud_sql_proxy",
              "-instances=company-2:europe-west3:company-mysql-1=tcp:3306",
              "-verbose=false"]
    securityContext:
      readOnlyRootFilesystem: true
      allowPrivilegeEscalation: false
      privileged: false
      runAsNonRoot: true

Cloud MySQL Docs snippet:

While the Cloud SQL Auth proxy can listen on any port, it creates outgoing or egress connections to your Cloud SQL instance only on port 3307. Because Cloud SQL Auth proxy calls APIs through the domain name sqladmin.googleapis.com, which does not have a fixed IP address, all egress TCP connections on port 443 must be allowed. If your client machine has an outbound firewall policy, make sure it allows outgoing connections to port 3307 on your Cloud SQL instance's IP.

EDIT 2:

I see this now:

2022/07/22 11:12:33 error checking scopes: *url.Error Get "http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes": dial tcp 169.254.169.254:80: i/o timeout | Get

Not sure what it is and allow port 80 would not be so nice I guess.

EDIT 3:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: {{ template "name" . }}
spec:
  podSelector:
    matchLabels:
      app: {{ template "name" . }}
  policyTypes:
  - Egress
  egress:
  - ports:
    - port: 53
      protocol: UDP
    - port: 53
      protocol: TCP
    - port: 443
      protocol: TCP
    - port: 3307
      protocol: TCP
    - port: 3306
      protocol: TCP
  - to:
    - ipBlock:
        cidr: 169.254.169.254/32

I still get the error, am I doing somthing wrong?

url.Error Get "http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/scopes

Edit 4:

kubectl describe NetworkPolicy network-p-3xl2j4


Name:         network-p-3xl2j4
Namespace:    develop
Created on:   2022-07-22 14:43:04 +0200 CEST
Labels:       app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: network-p-3xl2j4
              meta.helm.sh/release-namespace: develop
Spec:
  PodSelector:     app=network-p-3xl2j4
  Not affecting ingress traffic
  Allowing egress traffic:
    To Port: 53/UDP
    To Port: 53/TCP
    To Port: 443/TCP
    To Port: 3307/TCP
    To Port: 3306/TCP
    To: <any> (traffic not restricted by destination)
    ----------
    To Port: <any> (traffic allowed to all ports)
    To:
      IPBlock:
        CIDR: 169.254.169.254/32
        Except:
  Policy Types: Egress

Answer 1

The root of the problem was that I am using workload identity.

If you use network policy with GKE Workload Identity, you must allow egress to the following IP addresses and port numbers so your Pods can communicate with the GKE metadata server. For clusters running GKE version 1.21.0-gke.1000 and later, allow egress to 169.254.169.252/32 on port 988. For clusters running GKE versions prior to 1.21.0-gke.1000, allow egress to 127.0.0.1/32 on port 988. To avoid disruptions during auto-upgrades, allow egress to all of these IP addresses and ports.

So here is my minimal solution with all Ports needed.

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: {{ template "name" . }}
spec:
  podSelector:
    matchLabels:
      app: {{ template "name" . }}
  policyTypes:
    - Egress
  egress:
    - ports:
      - port: 53
        protocol: UDP
      - port: 53
        protocol: TCP
      - port: 443
        protocol: TCP
      - port: 3307
        protocol: TCP
    - to:
      - ipBlock:
          cidr: 169.254.169.252/32
      ports:
        - protocol: TCP
          port: 988

• 53 TCP/UPD for DNS
• 443 for calling sqladmin.googleapis.com Cloud SQL Docs
• 988 for Workload Identity
• 3307 for Cloud MySQL