Search code examples
azurekubernetesazure-pipelinesazure-akskubernetes-secrets

Trying to pass secret from Azure KeyVault via Azure pipeline as env var to an app deployed on AKS

I am trying to get a secret named "ApplicationInsightsInstrumentationKey" from an Azure Key Vault and make it available as environment variable to the well-known Microsoft Weather Forecast example ASP.Net Core application, which I deploy to AKS.

So in the Azure pipeline file I have these 2 tasks (the "Namespace" pipeline parameter value is "ccg-afarber-v3" and I use it for the overall secret name too. And the Key Vault is "ccg-afarber-v3-kv"):

- task: AzureKeyVault@2
  displayName: Read secrets from KV
  inputs:
    azureSubscription: $(armConnection)
    KeyVaultName: '${{ parameters.Namespace }}-kv'
    SecretsFilter: '*'
    RunAsPreJob: false

- task: AzureCLI@2
  displayName: Set secrets in AKS
  inputs:
    azureSubscription: $(armConnection)
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az aks get-credentials --resource-group ccg-config --name ccg-config-cluster
      kubectl delete 'secrets/${{ parameters.Namespace }}'
      kubectl create secret generic ${{ parameters.Namespace }} --from-literal=ApplicationInsightsInstrumentationKey=$(ApplicationInsightsInstrumentationKey)

The tasks succeed and I can see the secret in the AKS:

azure screenshot

After that I try to deploy the app via the following pipeline task:

- task: KubernetesManifest@0
  displayName: Deploy to AKS
  inputs:
    action: deploy
    namespace: ${{ parameters.Namespace }}
    kubernetesServiceConnection: $(aksConnection)
    manifests: |
      $(Build.ArtifactStagingDirectory)/manifests/deployment.yml
      $(Build.ArtifactStagingDirectory)/manifests/service.yml
    containers: |
      $(containerRegistry)/$(imageRepository):$(imageTag)

The task is using the deployment.yml file listed below:

metadata:
  name: exampleapplication 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: exampleapplication
  template:
    metadata:
      labels:
        app: exampleapplication 
    spec:
      containers:
        - name: exampleapplication 
          image: ccgconfigregistry.azurecr.io/exampleapplication:3a5e80a684c83e8037c2ccb07a45c8345501d154
          ports:
          - containerPort: 80
          env:
          - name: ApplicationInsightsInstrumentationKey
            valueFrom:
              secretKeyRef:
                name: ccg-afarber-v3
                key: ApplicationInsightsInstrumentationKey
                optional: false

Unfortunately, the deployment is stuck with:

# kubectl get pod --namespace ccg-afarber-v3
NAME                                 READY   STATUS                       RESTARTS   AGE
exampleapplication-c76448f49-hds2s   0/1     CreateContainerConfigError   0          19m

The kubectl describe pod --namespace ccg-afarber-v3 command prints:

Events:
  Type     Reason     Age                 From               Message
  ----     ------     ----                ----               -------
  Normal   Scheduled  20m                 default-scheduler  Successfully assigned ccg-afarber-v3/exampleapplication-c76448f49-hds2s to aks-nodepool1-84765107-vmss000003
  Warning  Failed     18m (x12 over 20m)  kubelet            Error: secret "ccg-afarber-v3" not found
  Normal   Pulled     17s (x97 over 20m)  kubelet            Container image "ccgconfigregistry.azurecr.io/exampleapplication:3a5e80a684c83e8037c2ccb07a45c8345501d154" already present on machine

I keep re-reading the docs:

but I don't understand, why is secret "ccg-afarber-v3" not found.

Solution

  • I have found my mistake - I was creating the Kubernetes secret in the "default" namespace, while my app was trying to retrieve it in the "ccg-afarber-v3" namespace.

    The following pipeline task with the added --namespace=ccg-afarber-v3 parameters has resolved my problem:

    - task: AzureCLI@2
  displayName: Set secrets in AKS
  inputs:
    azureSubscription: $(armConnection)
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az aks get-credentials --resource-group ccg-config --name ccg-config-cluster
      kubectl delete 'secrets/${{ parameters.Namespace }}' --namespace=${{ parameters.Namespace }}
      kubectl create secret generic ${{ parameters.Namespace }} --from-literal=ApplicationInsightsInstrumentationKey=$(ApplicationInsightsInstrumentationKey) --namespace=${{ parameters.Namespace }}