I have an Azure question. I use terraform in the Azure Cloud service. I try to start up 2 AKS cluster there. But I always get an error that my CIDR settings are wrong.
I use in Cluster one:
resource "azurerm_subnet" "cluster1-node-pool-subnet" {
name = "cluster1-node-pool-subnet"
resource_group_name = azurerm_virtual_network.cluster-vnet.resource_group_name
virtual_network_name = azurerm_virtual_network.cluster-vnet.name
address_prefixes = ["10.0.1.0/19"]
}
resource "azurerm_subnet" "cluster1-execution-nodes-subnet" {
name = "cluster1-execution-nodes-subnet"
resource_group_name = azurerm_virtual_network.cluster-vnet.resource_group_name
virtual_network_name = azurerm_virtual_network.cluster-vnet.name
address_prefixes = ["10.0.33.0/19"]
}
resource "azurerm_subnet" "cluster1-gpu-nodes-subnet" {
count = var.gpuNodePool ? 1 : 0
name = "execution-nodes-subnet"
resource_group_name = azurerm_virtual_network.cluster-vnet.resource_group_name
virtual_network_name = azurerm_virtual_network.cluster-vnet.name
address_prefixes = ["10.0.48.0/20"]
}
network_profile {
network_plugin = "azure"
service_cidr = "10.0.65.0/19"
dns_service_ip = "10.0.65.10"
docker_bridge_cidr = "172.17.0.1/16"
}
and in Cluster two:
resource "azurerm_subnet" "default-node-pool-subnet" {
name = "default-node-pool-subnet"
resource_group_name = azurerm_virtual_network.cluster-vnet.resource_group_name
virtual_network_name = azurerm_virtual_network.cluster-vnet.name
address_prefixes = ["10.0.0.0/19"]
}
resource "azurerm_subnet" "execution-nodes-subnet" {
name = "execution-nodes-subnet"
resource_group_name = azurerm_virtual_network.cluster-vnet.resource_group_name
virtual_network_name = azurerm_virtual_network.cluster-vnet.name
address_prefixes = ["10.0.32.0/19"]
}
resource "azurerm_subnet" "gpu-nodes-subnet" {
count = var.gpuNodePool ? 1 : 0
name = "execution-nodes-subnet"
resource_group_name = azurerm_virtual_network.cluster-vnet.resource_group_name
virtual_network_name = azurerm_virtual_network.cluster-vnet.name
address_prefixes = ["10.0.48.0/20"]
}
network_profile {
network_plugin = "azure"
service_cidr = "10.0.64.0/19"
dns_service_ip = "10.0.64.10"
docker_bridge_cidr = "172.17.0.1/16"
}
Azur now tell me that the prefix is wrong.
│ Error: creating Subnet: (Name "cluster1-node-pool-subnet" / Virtual Network Name "cluster-vnet" / Resource Group "cluster-infra-network"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidCIDRNotation" Message="The address prefix 10.0.1.0/19 in resource /subscriptions/xxx/resourceGroupscluster-infra-network/providers/Microsoft.Network/virtualNetworks/cluster-vnet/subnets/cluster1-node-pool-subnet has an invalid CIDR notation. For the given prefix length, the address prefix should be 10.0.0.0/19." Details=[]
│
│ with azurerm_subnet.cluster1-node-pool-subnet,
│ on k8s-rtc.tf line 7, in resource "azurerm_subnet" "cluster1-node-pool-subnet":
│ 7: resource "azurerm_subnet" "cluster1-node-pool-subnet" {
│
╵
╷
│ Error: creating Subnet: (Name "cluster1-execution-nodes-subnet" / Virtual Network Name "cluster-vnet" / Resource Group "cluster-infra-network"): network.SubnetsClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidCIDRNotation" Message="The address prefix 10.0.33.0/19 in resource /subscriptions/xxx/resourceGroups/cluster-infra-network/providers/Microsoft.Network/virtualNetworks/cluster-vnet/subnets/cluster1-execution-nodes-subnet has an invalid CIDR notation. For the given prefix length, the address prefix should be 10.0.32.0/19." Details=[]
│
│ with azurerm_subnet.cluster1-execution-nodes-subnet,
│ on k8s-rtc.tf line 14, in resource "azurerm_subnet" "cluster1-execution-nodes-subnet":
│ 14: resource "azurerm_subnet" "cluster1-execution-nodes-subnet" {
In my mind the CIDR and prefix are valid. any idea what is wrong?
There are basically two issues in your subnet definitions:
Azure tells you that you are using invalid CIDR notations. E.g. in case of cluster1-node-pool-subnet
you are specifying in address prefix ["10.0.1.0/19"]
. While 10.0.1.0/19
is a valid IP address, it belongs to the subnet 10.0.0.0/19
and Azure insists that you need to specify the address prefix based on the network address of the subnet.
The same applies to e.g. 10.0.33.0/19
which belongs to the subnet 10.0.32.0/19
.
This unveils the second issue, which Azure will report to you once the first is fixed: In both subnets you are using the same address prefixes in the subnets you are defining. To overcome this you could use e.g. ["10.0.0.0/19"]
for cluster1-node-pool-subnet
and ["10.1.0.0/19"]
for default-node-pool-subnet
and so on.