Search code examples
mongodbdockerdocker-composereplicasetmongodb-replica-set

How to seed a MongoDB single node replica set when the container starts


I was using a standalone MongoDB instance and had the following:

Docker Compose:

  database:
    build:
      context: .
      dockerfile: ./db/Dockerfile.mongodb
    ports:
      - 12000:27017
    volumes:
      - ./data/data:/data/db
    environment:
      - MONGO_INITDB_DATABASE=DB

./db/Dockerfile.mongodb:

FROM mongo:latest

COPY ./db/seed /import
COPY ./db/import/import.sh /docker-entrypoint-initdb.d/
RUN chmod 777 /docker-entrypoint-initdb.d/import.sh

./db/import/import.sh:

#!/bin/bash
for f in /import/*.json
do
    name=$(basename $f | cut -d'.' -f1)
    mongoimport --db=DB --collection=$name --jsonArray --file=$f
done

This would seed the database with .json files from ./db/seed/ directory whenever the container was started. Once it was up, my database would be in a ready to use state (for development, running automated tests, etc)

The problem with that setup is that I cannot use transactions in MongoDB.

Afterwards, I setup a single node replica set (to enable transactions) using the following:

Docker Compose:

services:
  mongo:
    build:
      context: .
      dockerfile: ./db/Dockerfile.mongodb
    command: [--replSet, rs0, --bind_ip_all, --port, "12000"]
    ports:
      - 12000:12000
    environment:
      - MONGO_INITDB_DATABASE=DB
    healthcheck:
      test: test $$(mongosh --port 12000 --quiet --eval "try {rs.initiate({_id:'rs0',members:[{_id:0,host:\"mongo:12000\"}]})} catch(e) {rs.status().ok}") -eq 1
      interval: 10s
      start_period: 0s

./db/Dockerfile.mongodb is same as above

./db/import/import.sh:

#!/bin/bash

for f in /import/*.json
do
    name=$(basename $f | cut -d'.' -f1)
    mongoimport --host=rs0/0.0.0.0:12000 --db=DB --collection=$name --jsonArray --file=$f
done

It took a lot of experimentation with the docker compose to even get the single node replica set working, because a lot of examples on the internet do not work. The ones that do work seem to all use the healthcheck mechanism to initialize the replica set since this gets run after container has been started. This causes a race condition with the import.sh script because the database is not setup as a replica set when the data load tries to run and it causes errors and the container stops.

How can I seed the replica set with data from .json files, without manually running mongoimport (or similar) commands after the container is up. I would like to be able to run the docker compose up and have the replica set db seeded and ready to go. I cannot find any examples of how this is done.

----------------------------UPDATE---------------------------------

Based on answer from @diego-freniche, I came up with a solution. I do not need a custom Dockerfile anymore (since that was used to just import the data in a non replica set standalone instance).

Docker Compose:

services:
  mongo:
    image: mongo:latest
    command: [--replSet, rs0, --bind_ip_all, --port, "12000"]
    ports:
      - 12000:12000
    environment:
      - MONGO_INITDB_DATABASE=DB
    healthcheck:
      test: test $$(mongosh --port 12000 --quiet --eval "try {rs.initiate({_id:'rs0',members:[{_id:0,host:\"mongo:12000\"}]})} catch(e) {rs.status().ok}") -eq 1
      interval: 10s
      start_period: 0s
  mongo_seeder:
    image: mongo:latest
    depends_on:
      database:
        condition: service_healthy
    volumes:
      - ./tools/db:/tmp/db/
    environment:
      - MONGO_INITDB_DATABASE=DB
      - MONGOIMPORT_HOST_STRING=rs0/database:12000
    command: bash /tmp/db/import/import.sh

./tools/db/import/import.sh

#!/bin/bash

for f in /tmp/db/seed/*.json
do
    name=$(basename $f | cut -d'.' -f1)
    mongoimport --host=$MONGOIMPORT_HOST_STRING --db=$MONGO_INITDB_DATABASE --collection=$name --jsonArray --file=$f
done

Solution

  • This is the docker-compose.yml I use it to start MongoDB Community Edition and run a script when the server is up and running.

    The key is that service mongodb_create_users depends on mongodb. So until mongodb is not fully loaded and the db running (I check that using healthcheck) mongodb_create_users does not start.

    In this case I run a mongosh script, but you can run any other import script you need. mongosh accepts a JS file with commands, the file create-mongodb-user.js is in my current directory (where this compose file is)

    services:
      # a MongoDB instance
      mongodb:
        image: mongodb/mongodb-community-server
        ports:
          - "27017:27017"
        environment:
          - MONGO_INITDB_ROOT_USERNAME=mongodb
          - MONGO_INITDB_ROOT_PASSWORD=password
        volumes:
          - type: bind
            source: ./data
            target: /data/db
          - ./:/tmp/import
        healthcheck:
          test: mongosh --eval 'db.hello()'
          interval: 10s
          timeout: 3s
          retries: 3
          start_period: 2s
        # mongodb://dvds:password@localhost:27017/dvds
      mongodb_create_users:
        image: mongodb/mongodb-community-server
        depends_on:
          mongodb:
            condition: service_healthy
        volumes:
          - ./:/tmp/import
        command: mongosh -u mongodb -p password --host mongodb admin -f /tmp/import/create-mongodb-user.js