How to Delete Azure FileShare Snapshots Along with Persistent Volume in Azure Kubernetes Services?

Question

I am working with Azure Kubernetes Services (AKS), where I have configured Persistent Volumes (PV) using Azure FileShares. Some of these FileShares have enabled backup functionality. When I delete a Persistent Volume Claim (PVC), I expect the associated Persistent Volume (PV) and FileShare to be deleted as well.

For FileShares without backup, the deletion process works as expected. However, for FileShares with backup enabled, I encounter an issue. When attempting to delete the PV associated with the PVC, I receive the following error:

storage.FileSharesClient#Delete: Failure sending request: StatusCode=409 --
  Original Error: autorest/azure: Service returned an error. Status=<nil>
  Code="DeleteShareWhenSnapshotLeased" Message="Unable to delete share because
  one or more share snapshots have active leases. Release the share snapshot
  leases or delete the share with the include-leased parameter for
  x-ms-delete-snapshots."

Despite deleting the PVC successfully, the PV remains in a "Released" state, and the associated FileShare is not deleted due to the active leases on snapshots.

I have verified that the FileShares have the soft-delete property enabled, and there is no lock preventing the editing of the resource group.

I have checked for any "Lease" properties in the snapshots or files inside but haven't found any that could be released to allow the deletion of the FileShare. It worked only after stopping the backup process and deleting backup data (in Azure Portal). The PV and FileShare were deleted without any additional actions.

Is there a way to automatically (and possibly force) delete all snapshots along with the FileShare when deleting a PV in AKS, even when there are active leases on the snapshots?

EDIT: kubernetes version is 1.27.7; StorageClass uses apiVersion: storage.k8s.io/v1; PersistentVolumeClaim and PersistentVolume uses apiVersion: v1.

Answer 1

In Azure Kubernetes Service, when you use Azure FileShare as the backing store for Persistent Volumes, and have backups enabled, you may run into issues deleting these file shares if they have snapshots due to active leases. By design, Azure does not automatically delete snapshots of a file share when the file share is deleted to prevent accidental data loss. This behavior is especially relevant when backup features are enabled, as these backups rely on snapshots.

When you delete a Persistent Volume Claim (PVC), Kubernetes sends a request to Azure to delete the corresponding file share. However, if there are any snapshots with active leases, Azure will block this request to prevent potential data loss, resulting in the PV remaining in the "Released" state without being fully deleted.

To address this issue, you can use Azure CLI or Azure Portal to manually delete snapshots before attempting to delete the PV.

 az storage share delete \
    --name arkoshare \
    --account-name arkostore \
    --snapshot <snapshot-id>

References:

• Questions about backing up Azure Files