I wish to bring a couple of containers using distinct compose.yml files instead of a single docker-compose.yml file. Within each docker-compose.<service>.yml file I have a Jinja template that fills out the the template file depending on which services are mentioned in a vars/main.yml
A simple example:
docker-compose.chronograf.yml.j2version: "3"
services:
chronograf:
image: chronograf:{{ services.chronograf.version }}
container_name: chronograf
{% if 'influxdb' in services %}
depends_on:
- influxdb
{% endif %}
environment:
- BASE_PATH=/chronograf
{% if 'influxdb' in services %}
- INFLUXDB_URL=http://influxdb:8086
{% endif %}
hostname: chronograf
labels:
- "traefik.enable=true"
- "traefik.http.routers.chronograf-router=chronograf-router@file"
- "traefik.http.routers.chronograf-router.service=chronograf@file"
security_opt:
- "no-new-privileges:true"
volumes:
- chronograf:/var/lib/chronograf
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
volumes:
chronograf:
docker-compose.influxdb.yml.j2version: "3"
services:
influxdb:
container_name: influxdb
image: influxdb:{{ services.influxdb.version }}
environment:
- INFLUXDB_HTTP_AUTH_ENABLED=true
- INFLUXDB_REPORTING_DISABLED=true
- INFLUXDB_DATA_QUERY_LOG_ENABLED=false
- INFLUXDB_HTTP_LOG_ENABLED=false
- INFLUXDB_CONTINUOUS_QUERIES_LOG_ENABLED=false
hostname: influxdb
security_opt:
- "no-new-privileges:true"
volumes:
- influxdb:/var/lib/influxdb
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
volumes:
influxdb:
the services above is defined as follows:
services:
chronograf:
version: 1.9-alpine
influxdb:
version: 1.8-alpine
Jinja will be able to add the depends_on part of the Compose YAML if services.influxdb exists (which it does)
within a playbook I have the following:
---
- host: all
tasks:
- name: Copy Services Compose Files
template:
src: "services/docker-compose.{{ item }}.yml.j2"
dest: "/home/test/influxstack/docker-compose.{{ item }}.yml"
with_items:
- "{{ services.keys() }}"
- name: Docker Compose with multiple Compose Files (v2)
community.docker.docker_compose:
project_name: influxstack
project_src: /home/test/influxstack
files:
- "docker-compose.{{ item }}.yml"
state: present
with_items:
- "{{ services.keys() }}"
Upon dry-running / running on the local machine (by creating a directory /home/test/influxstack I keep getting the following error:
TASK [pacedge : Docker Compose with multiple Compose Files (v2)]
ok: [localhost] => (item=influxdb)
failed: [localhost] (item=chronograf) => {"ansible_loop_var": "item", "changed": false, "item": "chronograf", "msg": "Configuration error - Service 'chronograf' depends on service 'influxdb' which is undefined."}
Ansible is unable to keep the sequence in order for files as mentioned in the docs
I have also tried adding:
{{ services.keys() | reverse(sort=True) }}
to try and bring InfluxDB up but I keep getting the same error
ansible [core 2.12.5]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/shantanoo/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/shantanoo/.local/lib/python3.8/site-packages/ansible
ansible collection location = /home/shantanoo/.ansible/collections:/usr/share/ansible/collections
executable location = /home/shantanoo/.local/bin/ansible
python version = 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0]
jinja version = 2.10.1
libyaml = True
Docker Compose version v2.6.1
pip list | grep -i "docker"
docker 4.1.0
docker-compose 1.25.0
dockerpty 0.4.1
You've got two separate docker-compose.yaml files; if you're starting them separately, you can't have dependencies between services in one file and services in another.
The way you've written your playbook, you are effectively doing this:
docker-compose -f docker-compose.chronograf.yaml up
docker-compose -f docker-compose.influxdb.yaml up
That's because you're using with_items loop: you're running the docker_compose task once for each in the list.
That won't work, because when docker-compose is bringing up the influxdb stack, it's not aware of the chronograf stack. The only way this can work is if you run docker-compose up referencing both files on the command line, as in:
docker-compose -f docker-compose.chronograf.yaml -f docker-compose.influxdb.yaml up
The equivalent change to your playbook would be:
- name: Docker Compose with multiple Compose Files (v2)
community.docker.docker_compose:
project_name: influxstack
project_src: /home/test/influxstack
files: "{{ compose_files }}"
state: present
vars:
compose_files: >-
{{ services.keys() |
map('regex_replace', '^(.*)$', 'docker-compose.\1.yaml')
}}
We've removed the loop, and we're instead constructing a list of
filenames and passing that as the files argument.
Tested using...
playbook.yaml:
- hosts: localhost
gather_facts: false
vars:
services:
service1:
service2:
tasks:
- name: Docker Compose with multiple Compose Files (v2)
community.docker.docker_compose:
project_name: influxstack
project_src: .
files: "{{ compose_files }}"
state: present
vars:
compose_files: >-
{{ services.keys() |
map('regex_replace', '^(.*)$', 'docker-compose.\1.yaml')
}}
docker-compose.service1.yaml:
version: "3"
services:
service1:
image: docker.io/alpine:latest
command:
- sleep
- inf
docker-compose.service2.yaml:
version: "3"
services:
service2:
image: docker.io/alpine:latest
command:
- sleep
- inf
Running:
ansible-playbook playbook.yaml
Results in:
$ docker-compose -p influxstack -f docker-compose.service1.yaml -f docker-compose.service2.yaml ps
Name Command State Ports
--------------------------------------------------
influxstack_service1_1 sleep inf Up
influxstack_service2_1 sleep inf Up