Search code examples
dockerdocker-composedocker-swarm

docker swarm container memory limit does not work

I'm experimenting with forcing a container to use more memory than it's allowed but I can't get it to work. The container is part of a stack defined with docker compose and it's deployed to docker in swarm mode.

Docker is allowing the container to go way above the 50M limit I've set. I was expecting docker to kill the container, throw an error, etc.

Can anyone help me on why Docker does not enforce the memory limit here?

The container in docker-compose.yml is defined to have a memory limit of 50M, and then I have setup a very simple PHP test which will try to allocate 200M. I've defined PHP mem limit to 128M.

This is my docker-compose.yml

  version: "3"

  services:
    nginx:
      image: nginx:latest
      restart: unless-stopped
      volumes:
        - ./deploy/nginx/nginx.conf:/etc/nginx/nginx.conf
        - ./public:/usr/share/nginx/html
      ports:
        - "8180:80"
      links:
        - app

    app:
      image: 127.0.0.1:5000/wpdemo
      build:
        context: .
        dockerfile: Dockerfile-app
      restart: unless-stopped
      volumes:
        - .:/var/www/html
      links:
        - mysql
      deploy:
        resources:
          limits:
            cpus: '0.50'
            memory: 50M
          reservations:
            cpus: '0.25'
            memory: 20M

    mysql:
        image: mysql:5.7
        restart: unless-stopped
        ports:
            - "13306:3306"
        environment:
            MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
        volumes:
            - ~/docker/volumes/mysql:/var/lib/mysql

Instead of docker killing the container, it allows it to take as much memory as it wants and PHP eventually stops the process throwing the error below:

"PHP message: PHP Fatal error: Allowed memory size of 125829120 bytes exhausted (tried to allocate 67108872 bytes) in /var/www/html/public/index.php on line 4"

I'm using Ubuntu 18.04. 

uname -a
Linux  4.18.10-041810-generic #201809260332 SMP Wed Sep 26 07:34:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Docker version 18.06.1-ce, build e68fc7a

docker-compose version 1.17.1, build unknown docker-py version: 2.5.1 CPython version: 2.7.15rc1 OpenSSL version: OpenSSL 1.1.0g 2 Nov 2017

This is the output of "docker stats" on the app container:

CONTAINER ID        NAME                                        CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
679c8495ac1d        stackdemo_app.1.hr3ufwlskhdafre39aqrshxyu   0.00%               43.81MiB / 50MiB    87.62%              106kB / 389kB       2.05GB / 10.6GB     5

This is the output of "docker info":

Containers: 36
 Running: 5
 Paused: 0
 Stopped: 31
Images: 450
Server Version: 18.06.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: active
 NodeID: wnegv5lp41wfs3epfrua489or
 Is Manager: true
 ClusterID: hq7o176yffjglxzb9pu3fiomr
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 192.168.1.120
 Manager Addresses:
  192.168.1.120:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: 69663f0bd4b60df09991c08812a60108003fa340
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.18.10-041810-generic
Operating System: Ubuntu 18.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.49GiB
Name: rafxps15
ID: QEX7:FEB3:J76L:DCAQ:SO4S:SWVE:4XPI:PI6R:YM4C:MV4I:C3PM:FLOQ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support
Solution

  • As you said in comment, swap is enabled on host but swap limit in cgroups does not supported yet.

    According to this enabling swap limit support. Note that reboot of system is essential.

    At last, —-memory-swap flag should be set. If you want to prevent your PHP app accessing swap, you should set it with the same value of —-memory. More details about memory swap settings.