Changing configuration in running Kubernetes Pod

Question

I have written the nifi.properties into a Kubernetes ConfigMap. When I deploy NiFi (as a StatefulSet) I want to have this nifi.properties file to be used by the NiFi I just deployed. To do so I added a volume for the ConfigMap and mounted it in the Container. The associated statefulset.yaml looks like this:

...
containers:
- name: 'myName'
  image: 'apache/nifi:latest'
  ports:
    - name: http
      containerPort: 8080
      protocol: TCP
    - name: http-2
      containerPort: 1337
      protocol: TCP
  volumeMounts:
    - name: 'nifi-config'
      mountPath: /opt/nifi/nifi-1.6.0/conf/nifi.properties
volumes:
- name: 'nifi-config'
  configMap:
    name: 'nifi-config'
...

This doesn't work, I think it is, because NiFi is already running and the nifi.properties file is locked by the service. The pod cannot be created, I get an error: ...Device or resource is busy. I also tried that with the bootstrap.conf file, which works, but I don't think that changes in there are recognized by the NiFi service because it would have to be restarted.

I already had the same issue with NiFi deployed on pure Docker, where I worked around by stopping the container, copying the files and starting the container; not very pretty, but working.

Using environment variables to change values in NiFi as stated here is also not an option, because the possibility of changing parameters there are very limited.

This problem doesn't occurs for NiFi only. I think that there are many situations where someone want's to change the configuration for a system running within Kubernetes, so I hope there is any solution to handle this issue.

Answer 1

I solved this with the help of this helm file, but changed it a bit. Actually it is nearly the same as the answer that pepov has given, but as stated in my comment, I got a CrashLoopBackOff. This also had nothing to do with the image version, because I used my own image that is based on NiFi 1.6.0 also containing some custom processors.

So my solution is to use the postStart handler of Kubernetes. Problem is that it is not guaranteed that this handler is called before the ENTRYPOINT (see). But in this case the pod would crash and restart, eventually getting it right; right now I haven't had this problem, so it seems to be good for now.
I copy the content of the configMap into a dedicated folder and copy them in the associated NiFi folder in the postStart handler.

So here is the statefulset.yaml:

...
containers:
- name: 'myName'
  image: 'apache/nifi:latest'
  ports:
    - name: http
      containerPort: 8080
      protocol: TCP
    - name: http-2
      containerPort: 1337
      protocol: TCP
  volumeMounts:
    - name: 'nifi-config'
      mountPath: /opt/nifi/nifi-1.6.0/kubeconfig
  lifecycle:
    postStart:
      exec:
        command:
          - bash
          - -c
          - |
            cp -a /opt/nifi/nifi-1.6.0/kubeconfig/. /opt/nifi/nifi-1.6.0/conf
volumes:
- name: 'nifi-config'
  configMap:
    name: 'nifi-config'
...