docker build ignore existing data and database config stored in my volume

Question

I have the following docker-compose.yml file:

# some other services here
......
......    

#############################
# Setup PostgreSQL container
#############################

  service-postgres:
    image: postgres:10-alpine
    container_name: service-postgres-server
    volumes:
      - ./data/postgres:/var/lib/postgresql
      - ./docker/initdb:/docker-entrypoint-initdb.d
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_PASSWORD=root
      - POSTGRES_DB=db_name

Each time I try to rebuild the containers running force-recreate I lose my database schema and data:

docker-compose up -d --force-recreate --build

But when I look under data/postgres I found that files and settings are there:

-$ ls data/postgres
PG_VERSION           pg_commit_ts         pg_ident.conf        pg_notify            pg_snapshots         pg_subtrans          pg_wal               postgresql.conf
base                 pg_dynshmem          pg_logical           pg_replslot          pg_stat              pg_tblspc            pg_xact              postmaster.opts
global               pg_hba.conf          pg_multixact         pg_serial            pg_stat_tmp          pg_twophase          postgresql.auto.conf postmaster.pid

How can I solve the issue and persist my database schema and data each time I need to rebuild the system.

Answer 1

Why the database data get lost?

I tried different things and compared the MySQL image with the PostgreSQL image. Now I found the problem. There is no difference using a named volume or a mounted volume but you have to bind the correct folder on the container to persist the database data:

services:
  service-postgres:
    image: postgres:10-alpine
    container_name: service-postgres-server
    volumes:
      - ./data/postgres/data:/var/lib/postgresql/data
      - ./docker/initdb:/docker-entrypoint-initdb.d
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: root
      POSTGRES_DB: db_name

You have to bind the /var/lib/postgresql/data folder to the host instead of the /var/lib/postgresql folder. You can also mount both folders to the host (the data folder and postgres folder) but you need the data folder to persist. On the MySQL image you only need the /var/lib/mysql folder to persist the data on the host.

Just for comparison: The MySQL image:

services:
  service-mysql:
    image: mysql:5.7
    container_name: service-mysql-server
    volumes:
      - ./data/mysql-db:/var/lib/mysql
      - ./docker/initdb:/docker-entrypoint-initdb.d
    ports:
      - "3306:3306"
    environment:
      MYSQL_PASSWORD: root
      MYSQL_DATABASE: db_name

With the above configuration the MySQL data is persisted on the host.

Using a named volume:

services:
  service-postgres:
    image: postgres:10-alpine
    container_name: service-postgres-server
    volumes:
      - postgresql-data:/var/lib/postgresql/data
      - ./docker/initdb:/docker-entrypoint-initdb.d
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: root
      POSTGRES_DB: db_name

volumes:
  postgresql-data:

With named volumes you can use docker-compose down -v to remove it. You can't remove a mounted volume with the docker-compose down command.

You can find the volume on the list of volumes with docker volume ls. You can also inspect the volume using docker volume inspect <volume-name> to get the mount point on the host machine.