Can docker-compose profiles be used together with docker-compose override to have a common code with configurable environments?

I have the requirement of having a unique docker-compose.yaml / infrastrucutre code which will be versioned across the different deployment stages.

I would like to have some ports exposed in development and not in production. As I learned from other questions this seems to not be possible (using the same .env file that is used to configure other environment variables for the containers).

My idea would be to have my docker-compose.yaml, for example:

version: "3.9"
services:
  myservice:
    image: myimage
    # **
    # configuration
    # **
    ports:
        - 80:80
        - 19980:19980

Then in production overrideit with a profile docker-compose.production.yaml

version: "3.9"
services:
  myservice:
    profiles:
      - production
    ports:
        - 80:80

This would allow to have always the same configuration (both .yamlf iles) and switch between them by just calling the docker-compose up command with the production profile (--profile).

My question is, does this work as expected or is the service always overwritten also when the profile flag is not provided?

Solution

Compose profiles only affect which services start; they do not have any effect on the options those services use. If you have multiple Compose files then the options in those files are merged according to a set of rules. My expectation is that this would take effect before the profile selection took place.

What you're describing seems like a fairly routine setup for multiple Compose files, without using the profile feature. The most common case I've seen is that a "development" setup strictly adds options to a "production" setup. In your example, both "production" and "development" publish port 80, but only development also publishes the debugger port. There also might be additional environment variables or bind mounts that only make sense in development, but you (usually) are not trying to remove values.

So in this setup, your base docker-compose.yml file would contain the production setup, with the minimum values that are used in all environments.

# docker-compose.yml
version: '3.8'
services:
  myservice:
    image: myimage
    ports:
      - '80:80'

Then you'd have a second file that only has the options that are added for the development setup:

# docker-compose.dev.yml / docker-compose.override.yml (see below)
version: '3.8'
services:
  myservice:
    # (do not need to repeat `image:`; could add `build:`)
    ports:
      - '19980:19980'

If you name the file docker-compose.override.yml, Compose will use both files automatically, and you need to make sure to push the base file but not the override file to the production environment.

# uses both files, if docker-compose.override.yml is present
docker-compose up -d

If you name it something else, you need to explicitly name all of the files with docker-compose -f options, on every Compose invocation.

docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
docker-compose -f docker-compose.yml -f docker-compose.dev.yml ps

(Or you can set the $COMPOSE_FILE environment variable, but you have to remember to set it in every shell session in every environment.)