PowerShell script for removing Key Vault secrets getting 409 Conflict errors

Question

I have written a PowerShell 5.1 script that removes secrets from an Azure Key Vault that has soft-delete enabled. I've learned that to successfully purge the secrets from such a Key Vault you have to:

1. Delete the secret.
2. Delete the secret again, this time specifying the -InRemovedState parameter.

My script is:

[CmdletBinding()]
Param(
    [Parameter(Mandatory = $True)]
    [string]$keyvaultName
)

#Remove assorted security items from Key Vault

# Shut up blather about deprecated features
 Set-Item -Path Env:\SuppressAzurePowerShellBreakingChangeWarnings -Value 'true'

Write-Output "Starting Removal"

### Remove Secrets
$secrets = Get-AzKeyVaultSecret -VaultName $keyvaultName

### Delete the secrets

foreach ($secret in $secrets) {
   Write-Output "Removing $($secret.Name)"
   Remove-AzKeyVaultSecret -VaultName $keyvaultName -Name $secret.Name -Force | Wait-Process

   if($?) {
      Write-Host "Removal of $($secret.Name) succeeded."
   }
   else {
      Write-Host "Removal of $($secret.Name) $vaultName failed."
   }
}

### Go back around and purge the secrets

foreach ($secret in $secrets) {
   Write-Output "Purging $($secret.Name)"
   Remove-AzKeyVaultSecret -VaultName $keyvaultName -Name $secret.Name -InRemovedState -Force | Wait-Process

   if($?) {
      Write-Host "Purge of $($secret.Name) succeeded."
   }
   else {
      Write-Host "Purge of $($secret.Name) $vaultName failed."
   }
}

Write-Output "Remove Complete"

However, when the script runs, occasionally I get "Conflict" errors. The number of errors that appear varies. I'm also getting a certificate showing up in the process out of nowhere, which I suspect is due to some sort of corruption in my Key Vault. Here's the output, where the script test-run.ps1 calls my purge script code shown above:

> .\test-run
Starting Removal
Removing QA-testing-certificate
Remove-AzKeyVaultSecret : Operation returned an invalid status code 'Forbidden'
At C:\Users\SESA280186\Desktop\OData\DevOps\Security\Key-Vault-material\keyvault-prototyping\remove-items2.ps1:25
char:4
+    Remove-AzKeyVaultSecret -VaultName $keyvaultName -Name $secret.Nam ...
+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Remove-AzKeyVaultSecret], KeyVaultErrorException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.RemoveAzureKeyVaultSecret

Removal of QA-testing-certificate  failed.
Removing QA-testing-secret
Removal of QA-testing-secret succeeded.
Removing QA-testing-secrets2
Removal of QA-testing-secrets2 succeeded.
Removing QA-testing-secrets3
Removal of QA-testing-secrets3 succeeded.
Purging QA-testing-certificate
Remove-AzKeyVaultSecret : Operation returned an invalid status code 'Forbidden'
At C:\Users\SESA280186\Desktop\OData\DevOps\Security\Key-Vault-material\keyvault-prototyping\remove-items2.ps1:39
char:4
+    Remove-AzKeyVaultSecret -VaultName $keyvaultName -Name $secret.Nam ...
+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Remove-AzKeyVaultSecret], KeyVaultErrorException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.RemoveAzureKeyVaultSecret

Purge of QA-testing-certificate  failed.
Purging QA-testing-secret
Remove-AzKeyVaultSecret : Operation returned an invalid status code 'Conflict'
At C:\Users\SESA280186\Desktop\OData\DevOps\Security\Key-Vault-material\keyvault-prototyping\remove-items2.ps1:39
char:4
+    Remove-AzKeyVaultSecret -VaultName $keyvaultName -Name $secret.Nam ...
+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Remove-AzKeyVaultSecret], KeyVaultErrorException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.RemoveAzureKeyVaultSecret

Purge of QA-testing-secret  failed.
Purging QA-testing-secrets2
Remove-AzKeyVaultSecret : Operation returned an invalid status code 'Conflict'
At C:\Users\SESA280186\Desktop\OData\DevOps\Security\Key-Vault-material\keyvault-prototyping\remove-items2.ps1:39
char:4
+    Remove-AzKeyVaultSecret -VaultName $keyvaultName -Name $secret.Nam ...
+    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [Remove-AzKeyVaultSecret], KeyVaultErrorException
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.KeyVault.RemoveAzureKeyVaultSecret

Purge of QA-testing-secrets2  failed.
Purging QA-testing-secrets3
Purge of QA-testing-secrets3 succeeded.
Remove Complete

Can anyone offer some advice on what is going on please? What have I overlooked?

Answer 1

Here is the reason you are seeing conflicts with some certificates.

Under the hood, key vault stores the private key of a certificate as a hidden secret object with the same name.

For example, if i create a certificate named "cert1" in my key vault, key vault creates a hidden secret with the name "cert1". Although this hidden secret is not visible to you in the Azure Portal, it does show up when you use powershell / CLI.

You are most likely trying to delete / purge a secret that still has a certificate associated with it, resulting in the conflict error.

Looks like you are using a for loop to iterate through all secrets which is likely the cause of this.