Calico Cloud - Egress domain network policy issue

Question

I am trying to configure egress traffic using domains via Calico Cloud.

I am aware that DNS feature is possible with paid Calico Enterprise or Calico Cloud as mentioned here. I am using calico cloud trial. I am able to access Egress traffic using destination.nets, however when I am using destination.domains it block whole egress traffic.

My Env:

GKE cluster: 1.20.15-gke.3400
Calico version: Calico Cloud (Already connected cluster, Network Policy created via Calico Cloud UI)
Firewalls: Disabled

For tests I'm using default nginx pod with installed ping. I've tried to ping or curl. To make it easier, I've allowed any protocol.

Working config with nets:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default.allow-google
spec:
  tier: default
  order: 0
  selector: app == "nginx"
  namespaceSelector: ''
  serviceAccountSelector: ''
  egress:
    - action: Allow
      source: {}
      destination:
        nets:
          - 142.250.185.132/32 ## ip of www.google.com
          - 87.248.100.216/32  ## ip of www.yahoo.com
  doNotTrack: false
  applyOnForward: false
  preDNAT: false
  types:
    - Egress

Issued policy using domains:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: default.allow-google
spec:
  tier: default
  order: 0
  selector: app == "nginx"
  namespaceSelector: ''
  serviceAccountSelector: ''
  egress:
    - action: Allow
      source: {}
      destination:
        domains:
          - '*.google.com'
          - '*.yahoo.com'
          - google.com
          - yahoo.com
  doNotTrack: false
  applyOnForward: false
  preDNAT: false
  types:
    - Egress

Questions:

1. Am I missing some basic configuration? It's related with tiers (I was using default and security tier but result was the same)?
2. Is there other option to allow egress policy based on DNS (open source/workarounds)?

Thanks in advance

Answer 1

I was lacking of kube-dns service policy.

When you connect your GKE cluster to Calico Cloud you don't have any policy. One you create nets rule, it will work as it works only on IPs. However, when you want to use FQDN/DNS you need to create policy to allow connectivity to kube-dns service. YAML would looks like below:

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-kube-dns
spec:
  selector: all()
  egress:
    - action: Allow
      destination:
        services:
          name: kube-dns
          namespace: kube-system

More details can be found here.

For troubleshooting you can check if you see any output using host ($ host www.google.com) command from dnsutils ($ apt install dnsutils).

After applying above YAML, destination.domains policy working as expected.