Search code examples
kubernetesminikubehashicorp-vaultvault

Environment variables concatenation issue with Vault agent-jnject in Kubernetes deployment


I would like your help to understand a scenario when using Vault's agent-inject in a deployment. Basically, I configured Vault in a Minikube cluster following this tutorial and then created the deployment as shown below:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: transaction-api
  namespace: tests
spec:
  replicas: 1
  selector:
    matchLabels:
      app: transaction-api
  template:
    metadata:
      labels:
        app: transaction-api
      annotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/agent-init-first: "true"
        vault.hashicorp.com/role: "role-transaction-api"
        vault.hashicorp.com/agent-inject-secret-transaction_database_config: "my-project/database/transaction-api/global"
        vault.hashicorp.com/agent-inject-secret-kafka_config: "my-project/messaging/kafka/global"
        vault.hashicorp.com/agent-inject-template-transaction_config: |
          {{- with secret "my-project/messaging/kafka/global" -}}
          export KafkaSettings__BootstrapServers="{{ .Data.data.bootstrapServers }}"
          export ENVIRONMENT_TESTS="{{ .Data.data.bootstrapServers }}" #to tests
          {{- end }}

          {{- with secret "my-project/database/transaction-api/global" -}}
          export ConnectionStrings__TransactionConnection="{{ .Data.data.connectionString }}"
          {{- end }}
    spec:
      serviceAccountName: default
      containers:
      - name: transaction-api
        image: leomarcamargo/transaction-api:k8s-test-v10
        command: ["/bin/sh"]
        args:
          - "-c"
          - ". /vault/secrets/transaction_config && printenv && dotnet Transactions.API.dll"
        env:
        # - name: ASPNETCORE_ENVIRONMENT
        #   value: "Development"
        - name: ASPNETCORE_URLS
          value: "http://+:7032"
        resources:
          requests:
            memory: "128Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        ports:
        - containerPort: 7032

The problem is that when I access the pod of the created deployment, it shows that the values ​​of the environment variables are being concatenated without proper separation, which is causing "export" to be added at the end of a variable. See:

export KafkaSettings__BootstrapServers="kafka.kafka.svc.cluster.local:9092"
export ENVIRONMENT_TESTS="kafka.kafka.svc.cluster.local:9092"export ConnectionStrings__TransactionConnection="Server=sql-server.database.svc.cluster.local,1433;Database=Transaction;User Id=sa;Password=Strong@Passw0rd;TrustServerCertificate=true;"

In this case, the ENVIRONMENT_TESTS variable was exactly like this:

ENVIRONMENT_TESTS=kafka.kafka.svc.cluster.local:9092export

The injection process seems to be correct, as no problem is reported in relation to this, but this concatenation is becoming a serious problem, as it has been damaging the values ​​of the respective environment variables. I have already worked on a project with agent-inject but in it the Kubernetes cluster and Vault configuration were already done and used agent-inject-template in this way and did not have these problems. However, now I am setting up the cluster from scratch and I am trying to understand what is happening. Searching on ChatGPT, he suggested inserting page breaks in each variable declaration in the agent-inject-template, something that in my opinion should not be correct.

So I would like your help to understand where the problem lies.


Solution

  • You are chomping (removing the new line with -}}) the new line. See helm chomping control

    This should work.

     {{- with secret "my-project/messaging/kafka/global" }}
     export KafkaSettings__BootstrapServers="{{ .Data.data.bootstrapServers }}"
     export ENVIRONMENT_TESTS="{{ .Data.data.bootstrapServers }}" #to tests
     {{- end }}
    
     {{- with secret "my-project/database/transaction-api/global" -}}
     export ConnectionStrings__TransactionConnection="{{ .Data.data.connectionString }}"
      {{- end }}