EDIT: It was a config error, I was setting wrong kv name :/
As said in title I'm facing an issue with secret creation using SecretProviderClass.
I've created my aks and my kv (and filled it) on azure. then I'll proceed to follow those steps using a user-assigned managed identity
but no secret
resource get created and pods got stuck on creation with mount failure.
those are the steps I followed
az extension add --name aks-preview
az extension update --name aks-preview
az aks enable-addons --addons azure-keyvault-secrets-provider -g $RESOURCE_GROUP -n $AKS_CLUSTER
az aks update -g $RESOURCE_GROUP -n $AKS_CLUSTER --enable-managed-identity --disable-secret-rotation
$AKS_ID = (az aks show -g $RESOURCE_GROUP -n $AKS_CLUSTER --query identityProfile.kubeletidentity.clientId -o tsv)
az keyvault set-policy -n $AZUREKEYVAULT --secret-permissions get --spn $AKS_ID
the SecretProviderClass manifest I'm using
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: azure-kvname
spec:
provider: azure
secretObjects:
- secretName: akvsecrets
type: Opaque
data:
- objectName: AzureSignalRConnectionString
key: AzureSignalRConnectionString
- objectName: BlobStorageConnectionString
key: BlobStorageConnectionString
- objectName: SqlRegistryConnectionString
key: SqlRegistryConnectionString
- objectName: TokenSymmetricKey
key: TokenSymmetricKey
parameters:
useVMManagedIdentity: "true"
userAssignedIdentityID: XXX # VMSS UserAssignedIdentity
keyvaultName: "sampleaks001" # the name of the KeyVault
objects: |
array:
- |
objectName: AzureSignalRConnectionString
objectType: secret
- |
objectName: BlobStorageConnectionString
objectType: secret
- |
objectName: SqlRegistryConnectionString
objectType: secret
- |
objectName: TokenSymmetricKey
objectType: secret
resourceGroup: sample # [REQUIRED for version < 0.0.4] the resource group of the KeyVault
subscriptionId: XXXX # [REQUIRED for version < 0.0.4] the subscription ID of the KeyVault
tenantId: XXX # the tenant ID of the KeyVault
and the deploy manifest
apiVersion: apps/v1
kind: Deployment
metadata:
name: trm-api-test
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: trm-api-test
template:
metadata:
labels:
app: trm-api-test
spec:
nodeSelector:
"kubernetes.io/os": linux
containers:
- name: trm-api-test
image: nginx
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 250m
memory: 256Mi
ports:
- containerPort: 80
env:
- name: AzureSignalRConnectionString
valueFrom:
secretKeyRef:
name: akvsecrets
key: AzureSignalRConnectionString
- name: TokenSymmetricKey
valueFrom:
secretKeyRef:
name: akvsecrets
key: TokenSymmetricKey
- name: BlobStorageConnectionString
valueFrom:
secretKeyRef:
name: akvsecrets
key: BlobStorageConnectionString
- name: SqlRegistryConnectionString
valueFrom:
secretKeyRef:
name: akvsecrets
key: SqlRegistryConnectionString
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets-store"
readOnly: true
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "azure-kvname"
---
apiVersion: v1
kind: Service
metadata:
name: trm-api-service-test
namespace: default
spec:
type: ClusterIP
selector:
app: trm-api-test
ports:
- port: 80
targetPort: 80
protocol: TCP
I'm sure I'm missing something, but can't understand what. Thanks in advance!
you are using the clientId, but it should be the objectId form the kubelet identity:
export KUBE_ID=$(az aks show -g <resource group> -n <aks cluster name> --query identityProfile.kubeletidentity.objectId -o tsv)
export AKV_ID=$(az keyvault show -g <resource group> -n <akv name> --query id -o tsv)
az role assignment create --assignee $KUBE_ID --role "Key Vault Secrets Officer" --scope $AKV_ID
This is a working SecretProviderClass i am using (adjusted to your config):
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: azure-kvname
spec:
provider: azure
secretObjects:
- data:
- objectName: AzureSignalRConnectionString
key: AzureSignalRConnectionString
- objectName: BlobStorageConnectionString
key: BlobStorageConnectionString
- objectName: SqlRegistryConnectionString
key: SqlRegistryConnectionString
- objectName: TokenSymmetricKey
key: TokenSymmetricKey
secretName: akvsecrets
type: Opaque
parameters:
usePodIdentity: "false"
useVMManagedIdentity: "true"
userAssignedIdentityID: XXX # Kubelet Client Id ( Nodepool Managed Idendity )
keyvaultName: "sampleaks001" # the name of the KeyVault
tenantId: XXX # the tenant ID of the KeyVault
objects: |
array:
- |
objectName: AzureSignalRConnectionString
objectAlias: AzureSignalRConnectionString
objectType: secret
- |
objectName: BlobStorageConnectionString
objectAlias: BlobStorageConnectionString
objectType: secret
- |
objectName: SqlRegistryConnectionString
objectAlias: SqlRegistryConnectionString
objectType: secret
- |
objectName: TokenSymmetricKey
objectAlias: TokenSymmetricKey
objectType: secret
You can also check documentation here as you will find better examples as on the Azure Docs.