Search code examples
node.jsdockerdocker-composedocker-machine

Docker in Docker

We have app and which will spin the short term (short span) docker containers. Right now, it runs in Ubunut16.04 server (VM) and we installed docker, and nodejs in same server. We have nodejs app which runs in same server so whenever a request comes in, then the nodejs app will spin up a docker container and execute a user input inside the docker container. Once after the docker finish its job or if it runs out of admin defined resources then the docker container will be forcefully killed (docker kill) and removed (docker rm).

Now my question is, is it best practices to run the Ubunte16.04 docker container and run nodjes app and the short term docker containers inside the Ubunuter16.04 docker container.

In short run a docker inside other docker container.

enter image description here

Solution

  • Docker-in-Docker is generally considered fragile and hard to maintain and using it isn’t a best practice. https://hub.docker.com/_/docker/ has a little discussion on this.

    A straightforward (but potentially dangerous) way to rearrange this is to give the server process access to the host’s Docker socket, with docker run -v /var/run/docker.sock:/var/run/docker.sock. Then it could launch its own Docker containers as needed. Note that if you do this, these sub-containers’ docker run -v options refer to the host’s filesystem, not the calling container’s filesystem, so if you’re trying to use the filesystem to transfer data this can get tricky. Also note that being able to run any Docker command this way gives unlimited access to the host, so you need to be extremely careful about how you launch containers.

    A larger redesign would be to introduce some sort of message-queueing system; I’ve successfully used RabbitMQ in the past but there are many other options. Instead of the server process launching a subprocess directly, it writes a message to a queue. Instead of the workers being short-lived processes that start and stop frequently, you have a long-lived worker that reads jobs off the queue and does them. This puts you in a much more established Docker space where nothing needs to dynamically start and stop containers, and you can easily test the Node-Rabbit-worker stack in a non-Docker environment.