Search code examples
dockerdocker-composevolumes

Docker change location of named volumes


I have a problem that I just can't understand. I am using docker to run certain containers, but I have problems with at least one Volume, where I't like to ask if anybody can give me a hint what I am doing wrong. I am using Nifi-Ingestion as example, but it affects even more container volumes.

First, let's talk about the versions I use:

  • Docker version 19.03.8, build afacb8b7f0
  • docker-compose version 1.27.4, build 40524192
  • Ubuntu 20.04.1 LTS

Now, let's show the volume in my working docker-compose-file:

In my container, it is configured as followed:

volumes:
    - nifi-ingestion-conf:/opt/nifi/nifi-current/conf

Below my docker-compose file it is defined as a normal named volume:

volumes:
    nifi-ingestion-conf:

This is a snippet from the docker-compose that I'd like to get working

In my container, it is configured in this case as followed (having my STORAGE_VOLUME_PATH defined as /mnt/storage/docker_data):

volumes:
    - ${STORAGE_VOLUME_PATH}/nifi-ingestion-conf:/opt/nifi/nifi-current/conf

On the bottom I guess there is something to do but I don't know what I could need to do here. In this case it is the same as in the working docker-compose:

volumes:
    nifi-ingestion-conf:

So, now whats my problem?

I have two docker-compose files. One uses the normal named volumes, and one uses the volumes in my extra mount path. When I run the containers, the volumes seem to work different since files are written in the first style, but not in the second. My mount paths are generated in the second version so there is nothing wrong with my environment variables in the .env-file.

Hint: the /mnt/storage/docker_data is an NFS-mount but my machine has the full privileges on that share.

Here is my fstab-entry to mount that volume (maybe I have to set other options):

10.1.0.2:/docker/data               /mnt/storage/docker_data        nfs     auto,rw

Bigger snippets

Here is a bigger snipped if the docker-compose (i need to cut and remove confident data, my problem is not that it does not work, it is only that the volume acts different. Everything for this one volume is in the code.):

version: "3"
services:
    nifi-ingestion:
        image: my image on my personal repo
        container_name: nifi-ingestion
        ports:
            - 0000
        labels:
            - app-specivic
        volumes:
            - ${STORAGE_VOLUME_PATH}/nifi-ingestion-conf:/opt/nifi/nifi-current/conf
            #working: - nifi-ingestion-conf:/opt/nifi/nifi-current/conf
        environment:
            - app-specivic
        networks:
            - cnetwork

volumes:
    nifi-ingestion-conf:

networks:
    cnetwork:
        external: false
        ipam:
            driver: default
            config:
                - subnet: 192.168.1.0/24

And here of the env (only the value we are using)

STORAGE_VOLUME_PATH=/mnt/storage/docker_data

Solution

  • if i understand your question correctly, you wonder why the following docker-compose snippet works for you

    version: "3"
    services:
      nifi-ingestion:
        volumes:
           - nifi-ingestion-conf:/opt/nifi/nifi-current/conf
    volumes:
      nifi-ingestion-conf:
    

    and the following docker-compose snippet does not work for you

    version: "3"
    services:
      nifi-ingestion:
        volumes:
          - ${STORAGE_VOLUME_PATH}/nifi-ingestion-conf:/opt/nifi/nifi-current/conf
    

    what makes them different is how you use volumes. you need to differentiate between mount host paths and mount named volumes

    You can mount a host path as part of a definition for a single service, and there is no need to define it in the top level volumes key.

    But, if you want to reuse a volume across multiple services, then define a named volume in the top-level volumes key.

    named volumes are managed by docker

    If you start a container with a volume that does not yet exist, Docker creates the volume for you.

    also, would advise you to read this answer

    update: you might also want to read about docker nfs volumes