I am trying to create 6 virtual machines in 3 availability sets for a 3-tier architecture. The DB virtual machine will have two NICs each for network segmentation, and the DB virtual machine will only have disk configuration. I have created terraform azure resources as per the below using the nested map of objects.
locals {
vms = {
nodes = {
app_node1 = {
"vm_name" = "app-poc"
"vm_num" = "1"
networks = {
nic1 = {
"vm_name" = "app-poc"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
}
"as" = "0"
},
app_node2 = {
"vm_name" = "app-poc"
"vm_num" = "2"
networks = {
nic1 = {
"vm_name" = "app-poc"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
}
"as" = "0"
},
service_node1 = {
"vm_name" = "service-poc"
"vm_num" = "1"
networks = {
nic1 = {
"vm_name" = "service-poc"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
}
"as" = "1"
},
service_node2 = {
"vm_name" = "service-poc"
"vm_num" = "2"
networks = {
nic1 = {
"vm_name" = "service-poc"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
}
"as" = "1"
},
db_node1 = {
"vm_name" = "db-poc"
"vm_num" = "1"
"as" = "2"
networks = {
nic1 = {
"vm_name" = "db-nic-1"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
nic2 = {
"vm_name" = "db-nic-2"
"subnet" = "/subscriptions/.../test-vnet/subnets/db"
}
}
disks = {
disk1 = {
"lun" = "0"
"size" = "1024"
},
disk2 = {
"lun" = "1"
"size" = "1024"
},
}
},
db_node2 = {
"vm_name" = "db-poc"
"vm_num" = "2"
"as" = "2"
networks = {
nic1 = {
"vm_name" = "db-nic-1"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
nic2 = {
"vm_name" = "db-nic-2"
"subnet" = "/subscriptions/.../test-vnet/subnets/app"
},
}
disks = {
disk1 = {
"size" = "1024"
"lun" = "0"
},
disk2 = {
"size" = "1024"
"lun" = "1"
},
}
},
}
}
}
Below is the terraform code create - Availability Sets, NICs, VMs, Managed Disk and Disk Attachment to Virtual Machine.
data "azurerm_resource_group" "rg" {
name = "test-rg"
}
resource "azurerm_availability_set" "as" {
count = length(var.availability_set_names)
name = var.availability_set_names[count.index]
location = data.azurerm_resource_group.rg.location
resource_group_name = data.azurerm_resource_group.rg.name
platform_fault_domain_count = 2
platform_update_domain_count = 5
}
resource "azurerm_network_interface" "nic-poc" {
for_each = {
for vm in flatten([
for vm_name, vm in local.vms.nodes : [
for nic_name, nic in vm.networks : {
vm_number = vm.vm_num,
vm_name = vm_name,
nic_value = nic.vm_name,
subnet_value = nic.subnet
nic_name = nic_name
}
]
]
) : "${vm.vm_name}-${vm.nic_name}" => vm
}
name = "${each.value.nic_value}-${each.value.vm_number}-nic"
location = data.azurerm_resource_group.rg.location
resource_group_name = data.azurerm_resource_group.rg.name
ip_configuration {
name = "${each.value.nic_name}-${each.value.vm_number}-ipconfig"
subnet_id = each.value.subnet_value
private_ip_address_allocation = "Dynamic"
}
}
resource "azurerm_linux_virtual_machine" "vm-poc" {
depends_on = [azurerm_availability_set.as, azurerm_network_interface.nic-poc]
for_each = local.vms.nodes
name = "${each.value.vm_name}-${each.value.vm_num}"
admin_username = "pluto-admin"
admin_password = "password@29"
disable_password_authentication = false
location = data.azurerm_resource_group.rg.location
resource_group_name = data.azurerm_resource_group.rg.name
network_interface_ids = [for nic_key, nic in azurerm_network_interface.nic-poc : nic.id]
size = "Standard_B2ms"
availability_set_id = azurerm_availability_set.as[each.value.as].id
identity {
type = "SystemAssigned"
}
os_disk {
name = "${each.value.vm_name}-${each.value.vm_num}-OSdisk"
caching = "ReadWrite"
storage_account_type = "Standard_LRS"
}
source_image_reference {
publisher = "RedHat"
offer = "RHEL"
sku = "82gen2"
version = "latest"
}
}
# Create managed disk
resource "azurerm_managed_disk" "db-data_disk" {
depends_on = [azurerm_linux_virtual_machine.vm-poc]
for_each = {
for vm in flatten([
for vm_name, vm in local.vms.nodes : [
for disk_name, disk in lookup(vm, "disks", {}) : {
vm_name = vm_name
disk_name = disk_name
disk_size = disk.size
vm_number = vm.vm_num
}
]
]
) : "${vm.vm_name}-${vm.disk_name}" => vm
}
name = "${each.value.vm_name}-${each.value.vm_number}-disk1"
location = data.azurerm_resource_group.rg.location
resource_group_name = data.azurerm_resource_group.rg.name
storage_account_type = "Premium_LRS"
create_option = "Empty"
disk_size_gb = each.value.disk_size
}
# Attach managed disk
resource "azurerm_virtual_machine_data_disk_attachment" "attach_disk" {
depends_on = [azurerm_managed_disk.db-data_disk]
for_each = {
for vm in flatten([
for vm_name, vm in local.vms.nodes : [
for disk_name, disk in lookup(vm, "disks", {}) : {
vm_name = vm_name
disk_lun = disk.lun
}
]
]
) : "${vm.vm_name}-${vm.disk_lun}" => vm
}
managed_disk_id = [for disk_key, disk in azurerm_managed_disk.db-data_disk : disk.id]
virtual_machine_id = [for vm_key, vm in azurerm_linux_virtual_machine.vm-poc : vm.id]
lun = each.value.disk_lun
caching = "ReadWrite"
}
variable "availability_set_names" {
type = list(string)
default = ["app-avset", "service-avset", "db-avset"]
}
Error #1 : Seems like the for loop is reading complete map of objects. azurerm_linux_virtual_machine - network_interface_ids = [for nic_key, nic in azurerm_network_interface.nic-poc : nic.id]
Error: creating Linux Virtual Machine (Subscription: "a94f0758-94a8-426d-a3b6-aec6e83b51de"
│ Resource Group Name: "test-rg"
│ Virtual Machine Name: "app-poc-1"): compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="NetworkInterfaceCountExceeded" Message="The number of network interfaces for virtual machine app-poc-1 exceeds the maximum allowed for the virtual machine size Standard_B2ms. The number of network interfaces is 8 and the maximum allowed is 3.
Error #2 at azurerm_virtual_machine_data_disk_attachment - managed_disk_id = [for disk_key, disk in azurerm_managed_disk.db-data_disk : disk.id]
virtual_machine_id = [for vm_key, vm in azurerm_linux_virtual_machine.vm-poc : vm.id]
Error: Invalid index
│
│ on main.tf line 213, in resource "azurerm_virtual_machine_data_disk_attachment" "attach_disk":
│ 213: managed_disk_id = azurerm_managed_disk.db-data_disk[each.key].id
│ ├────────────────
│ │ azurerm_managed_disk.db-data_disk is object with 4 attributes
│ │ each.key is "db_node2-0"
│
│ The given key does not identify an element in this collection value.
Can someone throw a light on how to retrieve the object values and read from one resource to another resource ?
Question Update with new error: Error 2.1 :
Planning failed. Terraform encountered an error while generating this plan.
╷
│ Error: Incorrect attribute value type
│
│ on main.tf line 215, in resource "azurerm_virtual_machine_data_disk_attachment" "attach_disk":
│ 215: managed_disk_id = [for disk_key, disk in azurerm_managed_disk.db-data_disk : disk.id if startswith(disk_key, "$each.key}-")]
│ ├────────────────
│ │ azurerm_managed_disk.db-data_disk is object with 4 attributes
│
│ Inappropriate value for attribute "managed_disk_id": string required.
Error1: Select only VM's specific interfaces
network_interface_ids = [for nic_key, nic in azurerm_network_interface.nic-poc : nic.id if startswith(nic_key, "${each.key}-")]
Error2:
you need to get a specific value from the map by key.
azurerm_managed_disk.db-data_disk created with for_each, so result is a map with [each.key] keys and resource as on object.
azurerm_virtual_machine_data_disk_attachment attaches one disk (managed_disk_id) to one VM (virtual_machine_id).
So:
add disk_name to each object in azurerm_managed_disk resource
fix
managed_disk_id = azurerm_managed_disk.db-data_disk[format("%s-%s", each.value.vm_name, each.value.disk_name)].id
virtual_machine_id = azurerm_linux_virtual_machine.vm-poc[each.value.vm_name].id