docker gitlab docker-registry gitlab-ci gitlab-ci-runner

Dial tcp i/o timeout with Docker in Gitlab CI build

I have an issue since a few days that I'm not able to understand, let aside fix.

I am working on the setup of a private Gitlab + Gitlab CI + Gitlab Docker registry environment to host and test my code and to build docker images from it in a runner.

I'm trying to build a Docker image in a Gitlab runner using the docker:dind image. I have the following error:

gitlab-ci-multi-runner 1.3.2 (0323456)
Using Docker executor with image docker:latest ...
Pulling docker image docker:dind ...
Starting service docker:dind ...
Waiting for services to be up and running...
Pulling docker image docker:latest ...
Running on runner-c682f650-project-23-concurrent-0 via 4b1582b2cffb...
Fetching changes...
HEAD is now at 7f10ed4 Finalize Docker and Gitlab CI workflow
Checking out 7f10ed4c as gitlab-ci-docker...
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.ademis.com:5005
Error response from daemon: Get https://registry.ademis.com:5005/v1/users/: dial tcp 46.105.98.95:5005: i/o timeout

I tried various things, including: - a ping to any IP works properly - port is open (when I try to login to another registry on port 5005 it works) - login from my local machine works properly

My architecture is full Docker on the server. I have three main containers (I have others, but not related to this issue):

jwilder/nginx-proxy            devserver_front_1
    0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp

gitlab/gitlab-ce:latest        devserver_gitlab_1
    80/tcp, 0.0.0.0:22->22/tcp, 0.0.0.0:5000->5000/tcp, 0.0.0.0:5005->5005/tcp, 443/tcp

gitlab/gitlab-runner:latest    devserver_runner_1

The HTTP access to Gitlab (port 80) go through jwilder/nginx-proxy but the Git access and the Docker registry access are direct.

When I launch the build, I use the Docker executor, thus a Docker container is built:

8ae8af78bc1b                   runner-c682f650-project-23-concurrent-0-cache-b36a40d3c05f7c6ff01d711a44f0d239

My problem seems to be inside that container. However, the usual suspects I had in mind (network and DNS) work well with other IP/domains, so I don't have any clue about the problem.

Any idea?

Solution

I suppose here your registry is running on the same Docker host, so my guess would be that communication from the host to the host itself on the public IP is broken, probably because of a bad iptables rules or something alike.