Search code examples
dockercentos7

Docker service start failed


I have a CentOS 7.2 VM with Docker installed. Docker service and Docker container worked normally previously. But when I tried to pull a docker image, the VM was shutdown abruptly. After I restarted the VM, the Docker service could not be started:

[root@AY13091717064020986bZ ~]# service docker start
Redirecting to /bin/systemctl start  docker.service
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.

systemctl status docker.service output:

[root@AY13091717064020986bZ ~]# systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2016-08-23 19:11:19 CST; 13min ago
     Docs: http://docs.docker.com
  Process: 1404 ExecStart=/usr/bin/docker-current daemon --exec-opt native.cgroupdriver=systemd $OPTIONS $DOCKER_STORAGE_OPTIONS $DOCKER_NETWORK_OPTIONS $ADD_REGISTRY $BLOCK_REGISTRY $INSECURE_REGISTRY (code=exited, status=1/FAILURE)
 Main PID: 1404 (code=exited, status=1/FAILURE)

Aug 23 19:11:17 AY13091717064020986bZ systemd[1]: Starting Docker Application Container Engine...
Aug 23 19:11:19 AY13091717064020986bZ docker-current[1404]: time="2016-08-23T19:11:19.448828158+08:00" level=warning msg="devmapper: Usage of loopback devices is strongly discou...v section."
Aug 23 19:11:19 AY13091717064020986bZ docker-current[1404]: time="2016-08-23T19:11:19.511103592+08:00" level=error msg="[graphdriver] prior storage driver \"devicemapper\" faile...t status 2"
Aug 23 19:11:19 AY13091717064020986bZ docker-current[1404]: time="2016-08-23T19:11:19.511196844+08:00" level=fatal msg="Error starting daemon: error initializing graphdriver: de...t status 2"
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: Failed to start Docker Application Container Engine.
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: Unit docker.service entered failed state.
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: docker.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

"journalctl -xe" output:

[root@AY13091717064020986bZ ~]# journalctl -xe
Aug 23 19:11:19 AY13091717064020986bZ kernel: device-mapper: block manager: btree_node validator check failed for block 146
Aug 23 19:11:19 AY13091717064020986bZ kernel: device-mapper: btree spine: node_check failed: csum 1600702373 != wanted 1600827965
Aug 23 19:11:19 AY13091717064020986bZ kernel: device-mapper: block manager: btree_node validator check failed for block 146
Aug 23 19:11:19 AY13091717064020986bZ kernel: Buffer I/O error on device dm-1, logical block 2621424
Aug 23 19:11:19 AY13091717064020986bZ docker-current[1404]: time="2016-08-23T19:11:19.511103592+08:00" level=error msg="[graphdriver] prior storage driver \"devicemapper\" failed: devmapper:
Aug 23 19:11:19 AY13091717064020986bZ docker-current[1404]: time="2016-08-23T19:11:19.511196844+08:00" level=fatal msg="Error starting daemon: error initializing graphdriver: devmapper: Base
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: Unit docker.service entered failed state.
Aug 23 19:11:19 AY13091717064020986bZ systemd[1]: docker.service failed.
Aug 23 19:11:19 AY13091717064020986bZ polkitd[1014]: Unregistered Authentication Agent for unix-process:1370:16052 (system bus name :1.22, object path /org/freedesktop/PolicyKit1/Authenticati
Aug 23 19:23:43 AY13091717064020986bZ systemd[1]: Starting Cleanup of Temporary Directories...
-- Subject: Unit systemd-tmpfiles-clean.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit systemd-tmpfiles-clean.service has begun starting up.
Aug 23 19:23:43 AY13091717064020986bZ systemd[1]: Started Cleanup of Temporary Directories.
-- Subject: Unit systemd-tmpfiles-clean.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit systemd-tmpfiles-clean.service has finished starting up.
--
-- The start-up result is done.

Docker version:

[root@AY13091717064020986bZ ~]# docker version
Client:
 Version:         1.10.3
 API version:     1.22
 Package version: docker-common-1.10.3-46.el7.centos.10.x86_64
 Go version:      go1.6.3
 Git commit:      d381c64-unsupported
 Built:           Thu Aug  4 13:21:17 2016
 OS/Arch:         linux/amd64
Cannot connect to the Docker daemon. Is the docker daemon running on this host?

Linux kernel version:

[root@AY13091717064020986bZ ~]# uname -a
Linux AY13091717064020986bZ 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@AY13091717064020986bZ ~]#

CentOS version:

[root@AY13091717064020986bZ ~]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID:    CentOS
Description:    CentOS Linux release 7.2.1511 (Core)
Release:    7.2.1511
Codename:    Core
[root@AY13091717064020986bZ ~]#

Solution

  • I had a similar issue. This was how I fixed it permanently:

    • Delete everything in /var/lib/docker. This will delete existing container and images:
    rm -rf /var/lib/docker
    
    • Then configure your daemon to use the "overlay" storage driver. Set the following flags in /etc/docker/daemon.json. If the file doesn't exist, just create it, and add the contents below:
    {
        "graph": "/mnt/docker-data",
        "storage-driver": "overlay"
    }
    

    Now start Docker normally again and all should work fine and always.

    See Start automatically at system boot.