Make Docker Compose Container wait for container PostgreSQL and restore

Question

I have a docker compose with one container PostgreSQL and another one with my App. I need to restore two databases into the PostgreSQL and then make the App start.

Currently I have tried this docker compose:

version: "3.7"
services:
  dataspace_postgres:
    container_name: custompostgres
    platform: linux/amd64
    build:
      context: ./
      dockerfile: Dockerfile.CustomPostgres
    networks:
      - dataspace
    environment:
      PGUSER: postgres
      POSTGRES_DB: postgres
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    ports:
      - 5434:5432
    volumes:
      - disk-postgres:/var/lib/postgresql/data
      - ./dumps/psql/managementreporting.sql:/managementreporting.sql
      - ./dumps/psql/idams_new.sql:/idams_new.sql
      - ./dumps/init.sh:/docker-entrypoint-initdb.d/init.sh
    healthcheck:
      test: ["CMD-SHELL", "sh -c 'pg_isready -U postgres -d managementreporing'"]
      interval: 10s
      timeout: 120s
      retries: 10

  dataspace_consolidatedreportingunit:
    container_name: consolidatedreportingunit
    platform: linux/amd64
    build:
      context: ./
      dockerfile: Dockerfile.ReportingTool
    networks:
      - dataspace
    ports:
      - 8080:80
      - 8081:443
    depends_on:
      dataspace_postgres:
        condition: service_healthy

networks:
  dataspace:
    driver: bridge

volumes:
  disk-postgres:
    driver: local

And my init.sh file:

pg_restore -v -c -d managementreporting managementreporting.sql
pg_restore -v -c -d idams_new idams_new.sql

Currently on startup it fails with the following error no matter what i tried:

pg_restore: connecting to database for restore custompostgres
| 2023-09-29 14:27:03.007 UTC [64] FATAL: database "managementreporting" does not exist custompostgres | pg_restore: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: database "managementreporting" does not exist custompostgres exited with code 1

For me the only way to restore the databases is to mount them directly into /docker-entrypoint-initdb.d/ (for dumps) but then I have no way to make the other container wait the postgesql to finish the restore.

Any ideas?

Answer 1

I managed to solve both of the issues, of restoring databases and waiting for the restores to complete using just simple methods.

First of all in the Docker-Compose, under volumes, you can restore the databases if you have backup them using plain mode by simply injecting them into docker-entrypoint-initdb.d folder, but using this method you cannot check when the restore actually completes, see below:

volumes:
  - disk-postgres:/var/lib/postgresql/data
  - ../PostgreSQL/dumps/plain/managementreporting.sql:/docker-entrypoint-initdb.d/managementreporting.sql
  - ../PostgreSQL/dumps/plain/idams_new.sql:/docker-entrypoint-initdb.d/idams_new.sql

Instead, we use custom format backups and we inject them into the root folder, so then we need a script to restore them and also to make a flag when the restore completes, see below:

volumes:
  - disk-postgres:/var/lib/postgresql/data
  - ../PostgreSQL/dumps/psql/managementreporting.sql:/managementreporting.sql
  - ../PostgreSQL/dumps/psql/idams_new.sql:/idams_new.sql
  - ../PostgreSQL/dumps/init.sh:/docker-entrypoint-initdb.d/init.sh
healthcheck:
  test: ["CMD-SHELL", "test -f /tmp/healthcheck.txt"]
  interval: 10s
  timeout: 120s
  retries: 10
  start_period: 60s

And the cmd-shell init.sh script for the restore and flag, see below

#Restore Consolidated Reporting Unit Database
createdb managementreporting #use it to avoid locale error on restore instead of pg_restore -C
pg_restore -v -U postgres -d managementreporting --no-owner /managementreporting.sql # -d should be postgres
#Restore Production BluePrint Idams New Database
createdb idams_new #use it to avoid locale error on restore instead of pg_restore -C
pg_restore -v -U postgres -d idams_new --no-owner /idams_new.sql # -d should be postgres
#Create a file to use healtcheck and ensure that PostgreSQL init has finished as during this action can accept connections
touch /tmp/healthcheck.txt #same file name at docker-compose healtcheck

In the init.sh script we use createdb to create the database instead of -C switch into the pg_restore as this produces problems when the database to be restored has different collation than default postgres database. (Also found that the -C switch expects to find the database already existing instead of creating the database, strange, most probably because expects to drop it first).

In addition, we use --no-owner switch into pg_restore in order to ignore previous owner from the server where we took the backup because this owner may not exist in the server which we use for restore.

In the end, we create a file (empty file) just to make it available for healtcheck in docker-compose and flag the container as healthy and then start the second container with the application. Finally do not pay so much attention into start_period: 60s, I placed this just there because I am sure the databases will not be restored before 60s and therefore there is no need to make tcp check calls before 60s.