Ansible: how to achieve idempotence with tasks that append files on host (w/o reverting to initial state)

Question

I am having a hard time getting to know how to create Ansible roles that are following the best practices according to documentation. The following use-case which I am looking at is e.g. enabling Filebeat on host. Filebeat can be configured by placing a module definition in /etc/filebeat/modules.d folder.

It works fine when I am adding modules. Idempotence is working, everytime, on each run of the role (playbook), a given set of modules is enabled.

But what I should do when I decide that a given module is not longer needed? I remove it from role, rerun a playbook, so that all other modules are enabled. But: the previous run enabled a module that I am not installing directly with role after changes. So my server state is still altered in a way that is different than the role is imposing itself.

My question is: should I take care of removing modules before I apply them so I always start from, let's say, fresh state?

E.g.:

- name: Remove modules
  file:
    dest: "/etc/filebeat/modules.d/{{ item }}"
    state: absent
  loop:
    - "module1.yml"
    - "module2.yml"
    - "module3.yml" # It was being installed in previous role, but not now

- name: Enable modules via 'modules.d' directory
  template:
    src: "modules.d/{{ item }}"
    dest: "/etc/filebeat/modules.d/{{ item }}"
    mode: '0644'
  loop:
    - "module1.yml"
    - "module2.yml"

So I remove module3.yml, because I remember that I've installed it before, and install module1.yml and module2.yml.

Instead of just installing what I need, no matter what has been installed before:

- name: Enable modules via 'modules.d' directory
  template:
    src: "modules.d/{{ item }}"
    dest: "/etc/filebeat/modules.d/{{ item }}"
    mode: '0644'
  loop:
    - "module1.yml"
    - "module2.yml"

Leaving me with module1.yml and module2.yml (desired) and, unfortunately: module3.yml (from previous role).

How to manage that to avoid such situations? And avoid treating server as one big stateful machine that even if I run a role, the output is different than desired, because something has been done before that I cannot see in current Ansible role code.

Do you code revert playbooks in your Ansible workflow to revert to initial state when needed?

I am curious. Thanks in advance for your reply.

Answer 1

In a nutshell:

- name: Configure filebeat modules
  hosts: all

  vars:
    fb_modules_d:
      - file: module1.yml
        state: present
      - file: module2.yml
        state: present
      - file: module3.yml
        state: absent

  tasks:
    - name: Make sure all needed module files are present
      template:
        src: "modules.d/{{ item.file }}"
        dest: "/etc/filebeat/modules.d/{{ item.file }}"
        mode: '0644'
      loop: "{{ fb_modules_d | selectattr('state', '==', 'present') }}"
      notifiy: restart_filebeat
 
    - name: Make sure all disabled modules are removed
      file:
        dest: "/etc/filebeat/modules.d/{{ item.file }}"
        state: "{{ item.state }}"
      loop: loop: "{{ fb_modules_d | selectattr('state', '==', 'absent') }}"
      notify: restart_filebeat

  handlers:
    - name: Restart filebeat service
      listen: restart_filebeat
      systemd:
        name: filebeat
        state: restarted

Note: I declared the variable inside the playbook for the example but that one one should most probably go inside your inventory (group or host level), and certainly not in a role (except in defaults for documentation)