fig up: docker containers start synchronisation

Question

For one of my home projects I decided to use docker containers and fig for orchestration (first time using those tools).

Here is my fig.yaml:

rabbitmq:
  image: dockerfile/rabbitmq:latest
mongodb:
  image: mongo
app:
  build: .
  command: python /code/app/main.py
  links:
   - rabbitmq
   - mongodb
  volumes:
   - .:/code

Rabbitmq starting time is much slower than loading time of my application. Even though rabbitmq container starts loading first (since it is in app links), when my app tries to connect to rabbitmq server it's not yet available (it's definately loading timing problem, since if I just insert sleep for 5 seconds before connecting to rabbitmq - everything works fine). Is there some standard way to resolve loading time synchronisation problems?

Thanks.

Answer 1

I don't think there is an standard way to solve this, but it is a known problem and some people have acceptable workarounds.

There is a proposal on the Docker issue tracker about not considering a container as started until it is listening at the exposed ports. However it likely won't be accepted due to other problems it would create elsewhere. There is a fig proposal on the same topic as well.

The easy solution is to do the sleep like @jcortejoso says. An example from http://blog.chmouel.com/2014/11/04/avoiding-race-conditions-between-containers-with-docker-and-fig/:

function check_up() {
    service=$1
    host=$2
    port=$3

    max=13 # 1 minute

    counter=1
    while true;do
        python -c "import socket;s = socket.socket(socket.AF_INET, socket.SOCK_STREAM);s.connect(('$host', $port))" \
        >/dev/null 2>/dev/null && break || \
        echo "Waiting that $service on ${host}:${port} is started (sleeping for 5)"

        if [[ ${counter} == ${max} ]];then
            echo "Could not connect to ${service} after some time"
            echo "Investigate locally the logs with fig logs"
            exit 1
        fi

        sleep 5

        (( counter++ ))
    done
}

And then use check_up "DB Server" ${RABBITMQ_PORT_5672_TCP_ADDR} 5672 before starting your app server, as described in the link above.

Another option is to use docker-wait. In your fig.yml.

rabbitmq:
  image: dockerfile/rabbitmq:latest
mongodb:
  image: mongo
rabbitmqready:
  image: aanand/wait
  links:
   - rabbitmq
app:
  build: .
  command: python /code/app/main.py
  links:
   - rabbitmqready
   - mongodb
  volumes:
   - .:/code