Difference in behavior when running docker-compose up command from CLI vs. Node.js script

Question

I have a Docker Compose file (docker-compose.yml) that defines two database containers and an Nginx container:

version: '3.1'
services:
  mysql1:
    image: mysql:8.0.21
    container_name: 'db1'
    restart: always
    environment:
      TZ: 'Asia/Jerusalem'
      MYSQL_ROOT_PASSWORD: '${DB_PASSWORD}'
    expose:
      - '3306'
    command: ["--server-id=1","--default-authentication-plugin=mysql_native_password"]
    security_opt:
      - seccomp:unconfined
    volumes:
      - ./db1/auto-generate:/auto-generate:rw
      - ./common/conf.d:/etc/mysql/conf.d:ro
      - ./common/initdb.d:/docker-entrypoint-initdb.d:ro
  mysql2:
    image: mysql:8.0.21
    container_name: 'db2'
    restart: always
    environment:
      TZ: 'Asia/Jerusalem'
      MYSQL_ROOT_PASSWORD: '${DB_PASSWORD}'
    expose:
      - '3306'
    command: ["--server-id=2","--default-authentication-plugin=mysql_native_password"]
    security_opt:
      - seccomp:unconfined
    volumes:
      - ./db2/auto-generate:/auto-generate:rw
      - ./common/conf.d:/etc/mysql/conf.d:ro
      - ./common/initdb.d:/docker-entrypoint-initdb.d:ro
  nginx:
    image: nginx:latest
    restart: always
    container_name: production_nginx
    ports:
      - '${PORT_DB1}:3001'
      - '${PORT_DB2}:3002'
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
    depends_on:
      - "mysql1"
      - "mysql2"

When I run the following command from the CLI, everything works as expected: sudo -E docker-compose -f docker-compose.yml up -d.

CLI output:

[+] Running 3/3
 ⠿ Container db2               Running                                                                                                                                                                                                 0.0s
 ⠿ Container db1               Running                                                                                                                                                                                                 0.0s
 ⠿ Container production_nginx  Started

However, when I run the same command from a Node.js script, it fails:

...
const { execSync } = require('child_process');

const switchCommand = `cd /opt/datateam  && sudo -E docker-compose  -f docker-compose.yml up -d`;

const password = config.db.password || dbDefaults.password;
const port = config.db.port || dbDefaults.port;
const { stagingPort } = config;

const getEnv = env => {
  return {
    PORT_DB1: env === 1 ? stagingPort : port,
    PORT_DB2: env === 2 ? stagingPort : port,
    DB_PASSWORD: password,
    COMPOSE_PROJECT_NAME: 'Mysql_dockers'
  };
};

...
await execSync(switchCommand, { env }); // <-- This fails

Node.js output:

Error: Command failed: cd /opt/datateam && sudo -E docker-compose -f docker-compose.yml up -d
16:02:38   Container db2  Creating
16:02:38   Container db1  Creating
16:02:38   Error response from daemon: Conflict. The container name "/db2" is already in use by container "78040292072bc3e959b563

The script is suppose to restart the nginx container, pointing to different ports (switching between them).

It appears that when running the command from the node.js script, it is if it doesn't see the containers already exist, and try to create them, contrary to docker compose up documentation which states that it should avoid creating containers that their configuration have not changed.

The following code is run on an EC-2 instance. Tried both from the default ec2-user and root.

It works on our on-prem linux server without issues.

Answer 1

The issue was using a different compose project name in a node.js parameter compared to the project name that was generated by docker implicitly.

When we ran docker-compose up for the first time, it ran from the /opt/datateam dir using CLI. Docker implicitly assigned it with a project name of "datateam" (the directory of the file). This was confirmed using docker inspect production_nginx | grep "compose.project".

The node.js file we ran executed the command sudo -E docker-compose -f /opt/datateam/docker-compose.yml up -d with an environment variable COMPOSE_PROJECT_NAME=Mysql_dockers. This caused docker to look for existing containers with a project name of Mysql_dockers, and since none exist, it attempted to create new ones, which ultimately caused the error to be thrown (Conflict. The container name "/db2" is already in use..).

The best solution was to alter the docker-compose.yml file and give it a name:

version: '3.1'
name: 'prod_mysql'

services:
...

And use the name in the environment variable we run the command with:

...
const getEnv = env => {
  return {
    PORT_DB1: env === 1 ? stagingPort : port,
    PORT_DB2: env === 2 ? stagingPort : port,
    DB_PASSWORD: password,
    COMPOSE_PROJECT_NAME: 'prod_mysql'
  };
};
...