Is it possible to mount a shared Azure disk in Azure Kubernetes to multiple PODs/Nodes?

I want to mount an Azure Shared Disk to the multiple deployments/nodes based on this: https://learn.microsoft.com/en-us/azure/virtual-machines/disks-shared

So, I created a shared disk in Azure Portal and when trying to mount it to deployments in Kubernetes I got an error:

"Multi-Attach error for volume "azuredisk" Volume is already used by pod(s)..."

Is it possible to use Shared Disk in Kubernetes? If so how? Thanks for tips.

Solution

Yes, you can, and the capability is GA.

An Azure Shared Disk can be mounted as ReadWriteMany, which means you can mount it to multiple nodes and pods. It requires the Azure Disk CSI driver, and the caveat is that currently only Raw Block volumes are supported, thus the application is responsible for managing the control of writes, reads, locks, caches, mounts, and fencing on the shared disk, which is exposed as a raw block device. This means that you mount the raw block device (disk) to a pod container as a volumeDevice rather than a volumeMount.

The documentation examples mostly points to how to create a Storage Class to dynamically provision the static Azure Shared Disk, but I have also created one statically and mounted it to multiple pods on different nodes.

Dynamically Provision Shared Azure Disk

Create Storage Class and PVC

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-csi
provisioner: disk.csi.azure.com
parameters:
  skuname: Premium_LRS  # Currently shared disk only available with premium SSD
  maxShares: "2"
  cachingMode: None  # ReadOnly cache is not available for premium SSD with maxShares>1
reclaimPolicy: Delete
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-azuredisk
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 256Gi  # minimum size of shared disk is 256GB (P15)
  volumeMode: Block
  storageClassName: managed-csi

Create a deployment with 2 replicas and specify volumeDevices, devicePath in Spec

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: deployment-azuredisk
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
      name: deployment-azuredisk
    spec:
      containers:
        - name: deployment-azuredisk
          image: mcr.microsoft.com/oss/nginx/nginx:1.17.3-alpine
          volumeDevices:
            - name: azuredisk
              devicePath: /dev/sdx
      volumes:
        - name: azuredisk
          persistentVolumeClaim:
            claimName: pvc-azuredisk

Use a Statically Provisioned Azure Shared Disk

Using an Azure Shared Disk that has been provisioned through ARM, Azure Portal, or through the Azure CLI.

Define a PersistentVolume (PV) that references the DiskURI and DiskName:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: azuredisk-shared-block
spec:
  capacity:
    storage: "256Gi" # 256 is the minimum size allowed for shared disk
  volumeMode: Block # PV and PVC volumeMode must be 'Block'
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  azureDisk:
    kind: Managed
    diskURI: /subscriptions/<subscription>/resourcegroups/<group>/providers/Microsoft.Compute/disks/<disk-name>
    diskName: <disk-name>
    cachingMode: None # Caching mode must be 'None'
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-azuredisk-managed
spec:
  resources:
    requests:
      storage: 256Gi
  volumeMode: Block
  accessModes:
    - ReadWriteMany
  volumeName: azuredisk-shared-block # The name of the PV (above)

Mounting this PVC is the same for both dynamically and statically provisioned shared disks. Reference the deployment above.