Search code examples
dockerdocker-swarmcircleci

Docker volumes with balance/services


I need to create several volumes for a specific container, however the mounts won't generate. I'm using the following example below:

         command: |
            ssh $SSH_USER_PROD@$DOCKER_IP_PROD -p $SSH_PORT_PROD '
            docker login -u "$LC_JFROG_USERNAME" -p "$LC_JFROG_PASSWORD" acessocard-docker.jfrog.io &&
            CREATE=$"docker service create --name $LC_SERVICE_NAME --replicas 1 --with-registry-auth $LC_SECRETS --network corporate --publish 9001:80 $LC_IMAGE:$LC_TAG --mount source=/var/bankreturn,target=/var/bankreturn --mount source=/var/bankreturn/error,target=/var/bankreturn/error --mount source=/var/bankreturn/input,target=/var/bankreturn/input --mount source=/var/bankreturn/processing,target=/var/bankreturn/processing --mount source=/var/bankreturn/success,target=/var/bankreturn/success --mount source=/var/bankreturn/temporary,target=/var/bankreturn/temporary" &&
            UPDATE=$"docker service update $LC_SERVICE_NAME --image $LC_IMAGE:$LC_TAG"&&
            docker service ls | grep $LC_SERVICE_NAME && $UPDATE || $CREATE'

Using "docker inspect", I see nothing is created inside Volumes section.

[
    {
        "Id": "eb8dcacc8c2f6b60dfcff783cfdbff0bdaf1114cd8df015b57324ae00a77965e",
        "Created": "2018-03-09T20:11:23.491659862Z",
        "Path": "dotnet",
        "Args": [
            "Acesso.BankReturn.Service.dll",
            "--mount",
            "source=/var/bankreturn,target=/var/bankreturn",
            "--mount",
            "source=/var/bankreturn/error,target=/var/bankreturn/error",
            "--mount",
            "source=/var/bankreturn/input,target=/var/bankreturn/input",
            "--mount",
            "source=/var/bankreturn/processing,target=/var/bankreturn/processing",
            "--mount",
            "source=/var/bankreturn/success,target=/var/bankreturn/success",
            "--mount",
            "source=/var/bankreturn/temporary,target=/var/bankreturn/temporary"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 2803,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2018-03-09T20:11:26.727751025Z",
            "FinishedAt": "0001-01-01T00:00:00Z"
        },
        "Image": "sha256:a16dc65817a7a0f3f3312bea0a659ca8fee26168053cb724365332ce48aa9739",
        "ResolvConfPath": "/var/lib/docker/containers/eb8dcacc8c2f6b60dfcff783cfdbff0bdaf1114cd8df015b57324ae00a77965e/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/eb8dcacc8c2f6b60dfcff783cfdbff0bdaf1114cd8df015b57324ae00a77965e/hostname",
        "HostsPath": "/var/lib/docker/containers/eb8dcacc8c2f6b60dfcff783cfdbff0bdaf1114cd8df015b57324ae00a77965e/hosts",
        "LogPath": "/var/lib/docker/containers/eb8dcacc8c2f6b60dfcff783cfdbff0bdaf1114cd8df015b57324ae00a77965e/eb8dcacc8c2f6b60dfcff783cfdbff0bdaf1114cd8df015b57324ae00a77965e-json.log",
        "Name": "/acesso-bankreturn.1.jqz53umombybbkpr0qpx6d4hv",
        "RestartCount": 0,
        "Driver": "aufs",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "default",
            "PortBindings": {},
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Name": "aufs",
            "Data": null
        },
        "Mounts": [],
        "Config": {
            "Hostname": "eb8dcacc8c2f",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "DOTNET_VERSION=2.0.5",
                "DOTNET_DOWNLOAD_URL=https://dotnetcli.blob.core.windows.net/dotnet/Runtime/2.0.5/dotnet-runtime-2.0.5-linux-x64.tar.gz",
                "DOTNET_DOWNLOAD_SHA=21D54E559C5130BB3F8C38EADACB7833EC90943F71C4E9C8FA2D53192313505311230B96F1AFEB52D74D181D49CE736B83521754E55F15D96A8756921783CD33",
                "ASPNETCORE_URLS=http://+:80",
                "ASPNETCORE_PKG_VERSION=2.0.5"
            ],
            "Cmd": [
                "--mount",
                "source=/var/bankreturn,target=/var/bankreturn",
                "--mount",
                "source=/var/bankreturn/error,target=/var/bankreturn/error",
                "--mount",
                "source=/var/bankreturn/input,target=/var/bankreturn/input",
                "--mount",
                "source=/var/bankreturn/processing,target=/var/bankreturn/processing",
                "--mount",
                "source=/var/bankreturn/success,target=/var/bankreturn/success",
                "--mount",
                "source=/var/bankreturn/temporary,target=/var/bankreturn/temporary"
            ],
            "Image": "acessocard-docker.jfrog.io/cico/bankreturn@sha256:cb0b34656178ae3aef7de88aa8f51e89c28c577cf8b79a0fe9b7c4e167ab02ff",
            "Volumes": null,
            "WorkingDir": "/app",
            "Entrypoint": [
                "dotnet",
                "Acesso.BankReturn.Service.dll"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.swarm.node.id": "6tn4fwh3wpfgrfj1swd9djw05",
                "com.docker.swarm.service.id": "xmerld0neauao8qbtoj61gd67",
                "com.docker.swarm.service.name": "acesso-bankreturn",
                "com.docker.swarm.task": "",
                "com.docker.swarm.task.id": "jqz53umombybbkpr0qpx6d4hv",
                "com.docker.swarm.task.name": "acesso-bankreturn.1.jqz53umombybbkpr0qpx6d4hv"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "689b5954ccf0239b7807e9dbbd81c93d59de88c51d53fdf78c8a69492861b023",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/689b5954ccf0",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "corporate": {
                    "IPAMConfig": {
                        "IPv4Address": "10.0.1.34"
                    },
                    "Links": null,
                    "Aliases": [
                        "eb8dcacc8c2f"
                    ],
                    "NetworkID": "weyjfrv37j9qy71l9cti1e7ca",
                    "EndpointID": "83aec692883a52492728e01b1e4fd58e2ba2a3e0626a3a40267bf506e14ae563",
                    "Gateway": "",
                    "IPAddress": "10.0.1.34",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:0a:00:01:22"
                },
                "ingress": {
                    "IPAMConfig": {
                        "IPv4Address": "10.255.0.35"
                    },
                    "Links": null,
                    "Aliases": [
                        "eb8dcacc8c2f"
                    ],
                    "NetworkID": "kk3agjaa27jci0q70efx74xzv",
                    "EndpointID": "57b0f1a85aba683133adfc653e2e0fd4fff11499b459641cfbae8a46e13c8867",
                    "Gateway": "",
                    "IPAddress": "10.255.0.35",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:0a:ff:00:23"
                }
            }
        }
    }
]

My company is using balance servers with Swarm and I noticed that volumes differ among them. So, servers are not able to view each others volumes.

Besides that, I also noticed that service is created in one server and container in another one. May this influence something? What should I do?

Thanks!


Solution

  • Several things:

    1. For Named Volumes, you don't want your source to be a host path. That's for bind-mounts. You'll want to add type=volume to the end of each mount option to be clear about what type you want, so: --mount source=myvolumename,target=/container/path,type=volume
    2. You've got options out of order, see docker service create --help at top for the order of things. Your $LC_IMAGE:$LC_TAG needs to be after all options (except for optional CMD, which it doesn't look like you need). Example: Example of correct service create would be docker service create --name myname --mount source=bankreturn,target=/var/bankreturn,type=volume imagename:tag notice image comes last.
    3. If you're using the default volume driver "local", a service's containers may be launched on nodes that don't have the volume data you're expecting. To fix, either use placement constraints to keep services on a specific node, or use a Swarm-aware shared storage volume driver, like REX-Ray.
    4. All swarm commands have to be done on a manager, and it will decide where best to put the service's tasks (containers) based on usage unless you use things like placement constraints, resource reservations, or other scheduling options in your service create command.