How to connect AKS to Postgres flexible server

Question

I have provisioned an AKS cluster and an Postgres Flexible server. I am able to schedule a deployment from my ACR however I am unable to connect to my DB from my Pod. What am I missing? I am trying to connect via myname.postgres.database.azure.com. I think both AKS and PG are on the same vnet. I am not sure exactly how the privatelink thing is meant to work, this could be the source of my confusion.

│ 2024/06/12 09:28:06 Unable to connect to database: failed to connect to `host=myname-server.postgres.database.azure.com user=user database=database`: hostname resolving error (lookup myname.postgres.database.azure.com on 10.0.0.10:53: no such host)

Core network module

# Resource Group
resource "azurerm_resource_group" "default" {
  name     = "${var.prefix}_rg"
  location = var.location
}

resource "azurerm_virtual_network" "default" {
  name                = "${var.prefix}-vnet"
  location            = var.location
  resource_group_name = azurerm_resource_group.default.name
  address_space       = ["10.0.0.0/16"]
}

resource "azurerm_network_security_group" "default" {
  name                = "${var.prefix}-nsg"
  location            = azurerm_resource_group.default.location
  resource_group_name = azurerm_resource_group.default.name

  security_rule {
    name                       = "test123"
    priority                   = 100
    direction                  = "Inbound"
    access                     = "Allow"
    protocol                   = "Tcp"
    source_port_range          = "*"
    destination_port_range     = "*"
    source_address_prefix      = "*"
    destination_address_prefix = "*"
  }
}

resource "azurerm_subnet" "default" {
  name                 = "${var.prefix}-subnet"
  virtual_network_name = azurerm_virtual_network.default.name
  resource_group_name  = azurerm_resource_group.default.name
  address_prefixes     = ["10.0.2.0/24"]
  service_endpoints    = ["Microsoft.Storage"]

  delegation {
    name = "fs"

    service_delegation {
      name = "Microsoft.DBforPostgreSQL/flexibleServers"

      actions = [
        "Microsoft.Network/virtualNetworks/subnets/join/action",
      ]
    }
  }
}

resource "azurerm_subnet_network_security_group_association" "default" {
  subnet_id                 = azurerm_subnet.default.id
  network_security_group_id = azurerm_network_security_group.default.id
}

resource "azurerm_private_dns_zone" "default" {
  name                = "${var.prefix}-pdz.postgres.database.azure.com"
  resource_group_name = azurerm_resource_group.default.name

  depends_on = [azurerm_subnet_network_security_group_association.default]
}

resource "azurerm_private_dns_zone_virtual_network_link" "default" {
  name                  = "${var.prefix}-pdzvnetlink.com"
  private_dns_zone_name = azurerm_private_dns_zone.default.name
  virtual_network_id    = azurerm_virtual_network.default.id
  resource_group_name   = azurerm_resource_group.default.name
}

resource "azurerm_postgresql_flexible_server" "default" {
  name                   = "${var.prefix}-server"
  resource_group_name    = azurerm_resource_group.default.name
  location               = azurerm_resource_group.default.location
  version                = "13"
  delegated_subnet_id    = azurerm_subnet.default.id
  private_dns_zone_id    = azurerm_private_dns_zone.default.id
  administrator_login    = "prop"
  administrator_password = var.db_password
  zone                   = "1"
  storage_mb             = 32768
  sku_name               = "GP_Standard_D2s_v3"
  backup_retention_days  = 7

  depends_on = [azurerm_private_dns_zone_virtual_network_link.default]
}

resource "azurerm_postgresql_flexible_server_database" "default" {
  name      = "${var.prefix}-db"
  server_id = azurerm_postgresql_flexible_server.default.id
  collation = "en_US.utf8"
  charset   = "UTF8"
}

AKS Module

# Create cluster
resource "azurerm_kubernetes_cluster" "aks" {
  name                = "${var.prefix}_aks"
  dns_prefix          = "${var.prefix}aks"
  location            = var.location
  resource_group_name = var.resource_group_name
  sku_tier            = "Free"

  default_node_pool {
    name                        = "${var.prefix}pool"
    node_count                  = var.node_count
    vm_size                     = var.vm_size
    temporary_name_for_rotation = "${var.prefix}tmp"
  }

  identity {
    type = "SystemAssigned"
  }

  network_profile {
    network_plugin      = "azure"
    network_plugin_mode = "overlay"
    ebpf_data_plane     = "cilium"
  }
}

resource "azurerm_container_registry" "acr" {
  name                = "${var.prefix}acr"
  location            = var.location
  resource_group_name = var.resource_group_name
  sku                 = "Basic"
  admin_enabled       = true
}

Update

kubectl exec -it busybox -- nslookup myname-server.postgres.database.azure.com                                                                                                           
Server:     10.0.0.10
Address:    10.0.0.10:53

Non-authoritative answer:
myname-server.postgres.database.azure.com   canonical name = e7657xxxxxx.myname-pdz.postgres.database.azure.com

Non-authoritative answer:
myname-server.postgres.database.azure.com   canonical name = e7657xxxxxx.myname-pdz.postgres.database.azure.com

and if I lookup the cname

** server can't find e7657xxxxxx.myname-pdz.postgres.database.azure.com: NXDOMAIN

Somehow the problem is that the AKS DNS has no way to resolve cname here. I have no idea how to resolve this cname from the cluster

Answer 1

│ 2024/06/12 09:28:06 Unable to connect to database: failed to connect to `host=myname-server.postgres.database.azure.com user=user database=database`: hostname resolving error (lookup myname.postgres.database.azure.com on 10.0.0.10:53: no such host)

The error you encountered above is due to DNS not forwarding the traffic to Azure DNS (168.63.129.16) because you are using a custom DNS server (10.0.0.10)

To resolve the issue, you need to configure the Azure DNS resolver to forward the traffic from your custom DNS server to Azure Private DNS (168.63.129.16)

By configuring the DNS resolver, AKS will accept traffic coming from the custom DNS server and resolve the address of your database. Refer to the MS Doc for creating Azure DNS Private Resolver.

Reference: What is Azure DNS Private Resolver?

Azure Private Resolver for virtual network and on-premises workloads