Setting this question up to self-answer because I lost too much time solving it, and I want to make sure the answer is put out there for future sufferers to find and avoid the same rabbit-hole I went down...
I'm trying to use the az network dns zone export command from the Az Cli in a PowerShell script to export the configuration from an Azure DNS Zone, but for some reason it's only exporting the first 100 records if I use az login to connect as a Service Principal with a custom IAM role assigned.
E.g.:
$ErrorActionPreference = "Stop";
Set-StrictMode -Version "Latest";
# see https://github.com/Azure/azure-cli/issues/26052
az config unset core.allow_broker
# log out any existing user
az logout
# log back in as the spn
az login --service-principal `
--username "my-client-id" `
--password "my-client-secret" `
--tenant "my-tenant-id"
# export the dns zone (only the first 100 records get exported :-(
az network dns zone export --resource-group "my-dns-rg" --name "mydnszone.com"
If I run the az network dns zone export command with my personal account logged in (which is an Owner on My Subscription) it works fine and exports all 575 records, so the problem appears to be permission-related, but it's confusing because the first 100 records are exported as the Service Principal so it has at least some permissions to read them.
My environment setup is as follows:
I have an Azure subscription My Subscription
In this subscription I have a public Azure DNS Zone mydnszone.com which has 575 zone records
I have a custom IAM role called My DNS Reader Role which has the following Microsoft.Network provider operations allowed:
| Operation | Description |
|---|---|
| Microsoft.Network/dnsZones/read | Get the DNS zone, in JSON format. The zone properties include tags, etag, numberOfRecordSets, and maxNumberOfRecordSets. Note that this command does not retrieve the record sets contained within the zone. |
| Microsoft.Network/dnsZones/recordsets/read | Gets DNS record sets across types |
I also have an Azure Service Principal my_dns_reader which has been assigned My DNS Reader Role IAM role scoped to My Subscription
Permissions required to export all pages of records via the az network dns zone export command are:
| Operation | Description |
|---|---|
| Microsoft.Network/dnszones/read | Get the DNS zone, in JSON format. The zone properties include tags, etag, numberOfRecordSets, and maxNumberOfRecordSets. Note that this command does not retrieve the record sets contained within the zone. |
| Microsoft.Network/dnszones/recordsets/read | Gets DNS record sets across types |
| Microsoft.Network/dnszones/all/read | Gets DNS record sets across types |
The Az Cli uses the Record Sets - List By Dns Zone Azure REST endpoint to export a dns zone with the az network dns zone export command.
This endpoint uses pagination (with a default page size of 100 recordsets) to return recordsets in a DNS zone. The Microsoft.Network/dnsZones/recordsets/read provider operation grants access to this endpoint.
The first page of results includes a nextLink which points to a different endpoint - Record Sets - List All By Dns Zone - for page 2 onwards, and this endpoint requires the Microsoft.Network/dnszones/all/read provider operation.
If this provider operation is missing from the current account the "Record Sets - List All By Dns Zone" endpoint returns paginated results but with zero rows of data in it.
Adding the Microsoft.Network/dnszones/all/read to my custom IAM role allowed the az network dns zone export command to export all recordsets in the zone.
Confusingly, no error messages are shown if the Microsoft.Network/dnsZones/recordsets/read provider operation is present but the Microsoft.Network/dnszones/all/read provider operation is missing - instead the "Record Sets - List All By Dns Zone" endpoint returns pagination responses but with zero records included in the data.
Running the az network dns zone export command with the --debug flag shows it first makes a call to the Record Sets - List By Dns Zone Azure REST endpoint - e.g.
https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/my-dns-rg/providers/Microsoft.Network/dnsZones/mydnszone.com/recordsets?api-version=2023-07-01-preview
If the current account has the Microsoft.Network/dnszones/recordsets/read provider operation this returns a response in the format:
{
"nextLink": "https://management.azure.com:443/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/my-dns-rg/providers/Microsoft.Network/dnszones/mydnszone.com/ALL?api-version=2023-07-01-preview&$skipToken=xxxxxxxxxxxxx",
"value": [
... first 100 recordets from the dns zone ...
]
}
(If there are less than 100 records the nextLink field is missing from the response).
Note that the nextLink ends in /ALL which is the Record Sets - List All By Dns Zone endpoint, so subsequent calls made by the Az Cli to read paginated data hit this endpoint instead of the original "Record Sets - List By Dns Zone".
Unfortunately, this endpoint requires a different provider operation - namely Microsoft.Network/dnszones/all/read - if this is missing the response is in the following format:
{
"nextLink": "https://management.azure.com:443/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/resourceGroups/my-dns-rg/providers/Microsoft.Network/dnszones/mydnszone.com/ALL?api-version=2023-07-01-preview&$skipToken=xxxxxxxxxxxxx",
"value": []
}
Note that pagination still works. For a zone with 575 records the nextLink redirects to a chain of 5 additional pages after the initial request, which is the correct number for a zone with 575 records, but the requests ending in /ALL contain "value": [].
When trying to create a minimal custom IAM role for exporting dns zones, be sure to add all the provider operations in the "tl;dr" section at the top of this answer.