How to tell terraform it should try to delete a soft-deleted azure keyvault key, when it exists?

Question

Azure enforces purge protection on all keyvault keys by default and this is causing some headaches when using terraform. Specicially this can run into a situation where you deleted a VM using a disk-encryption key and as a result, terraform deleted the disk-encryption key as well from the keyvault.

If later on you try to recreate that VM (and of course, you use the same encryption key name, as it's based on the VM name), the creation process will always fail

azurerm_key_vault_key.disk_encryption[5]: Creating...
╷
│ Error: Creating Key: keyvault.BaseClient#CreateKey: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="Conflict" Message="Key vmname-custom-disk-encryption is currently in a deleted but recoverable state, and its name cannot be reused; in this state, the key can only be recovered or purged." InnerError={"code":"ObjectIsDeletedButRecoverable"}
│ 
│   with azurerm_key_vault_key.disk_encryption[5],
│   on encryption.tf line 12, in resource "azurerm_key_vault_key" "disk_encryption":
│   12: resource "azurerm_key_vault_key" "disk_encryption" {

This would be easily resolved, if in this situation, terraform simply recovered the key in question before modifying it in-place.

Is there a way to do this, or otherwise to work around it?

Yes, I know I could theoretically create a VM with a different name, but in this instance, I am creating a VM as part of a cluster using a terraform count and using the count.index in the VM name. I run into this issue by testing if I can remove VM nodes from the cluster and then re-add them, but due to them keeping the same name, I run into the recoverable key terraform doesn't seem to handle.

EDIT: Through trial and error, I have discovered the following.

If I put the following in my azurerm provider

provider "azurerm" {
  features {
    key_vault {
      purge_soft_delete_on_destroy = false
    }
  }
}

The issue I reported will occur. The key will not be automatically recovered when found soft-deleted.

However if I put instead

provider "azurerm" {
  features {}
}

Then the key will be recovered and reused without problem, BUT terraform will fail when destroying because it will be trying to purge it as well!

│ Error: purging of Key "vmname-custom-disk-encryption" (Key Vault "https://keyvaultname.vault.azure.net/") : keyvault.BaseClient#PurgeDeletedKey: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="Forbidden" Message="Operation \"purge\" is not allowed because purge protection is enabled for this vault. Key Vault service will automatically purge it after 
the retention period has passed.\r\nVault: keyvaultname;location=westeurope"

I'll open a bug report as this seems like a bug to me (why would purge_soft_delete_on_destroy = false prevent key recovery?) but I'm curious to see if anyone has a workaround.

Answer 1

This is a workaround rather than an answer, but it might help someone else:

If you put the following in your azurerm provider

provider "azurerm" {
  features {
    key_vault {
      purge_soft_delete_on_destroy = false
    }
  }
}

The issue I reported will occur. The key will not be automatically recovered when found soft-deleted.

However if you put instead

provider "azurerm" {
  features {}
}

Then the key will be recovered and reused without problem, BUT terraform will fail when destroying because it will be trying to purge it as well!

│ Error: purging of Key "vmname-custom-disk-encryption" (Key Vault "https://keyvaultname.vault.azure.net/") : keyvault.BaseClient#PurgeDeletedKey: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="Forbidden" Message="Operation \"purge\" is not allowed because purge protection is enabled for this vault. Key Vault service will automatically purge it after 
the retention period has passed.\r\nVault: keyvaultname;location=westeurope"

With the second approach, you ignore failed destroy errors. The next destroy run should work without issues (as nothing will exist at that point)

I have opened a bug report for terraform azurerm provider about this