Search code examples
ansibleyaml

Define redundant task options centrally and reuse them in different tasks


Is it possible to define task options centrally to reuse them in different tasks?

I am currently working with several community.postgresql modules, like postgresql_db, postgresql_user and postgresql_privs. I always have to define the options login_host, port, login_user, login_password, but these are the same the whole execution, so I would like to define them once centrally and just include each.

So far my tasks look something like this:

- name: Create database
  community.postgresql.postgresql_db:
    name: "{{ pg_db_name }}"

    login_host: "{{ pg_host }}"
    port: "{{ pg_port }}"
    login_user: "{{ pg_admin.username }}"
    login_password: "{{ pg_admin.password }}"

- name: Create database user
  community.postgresql.postgresql_user:
    name: "{{ pg_user.username }}"
    password: "{{ pg_user.password }}"

    login_host: "{{ pg_host }}"
    port: "{{ pg_port }}"
    login_user: "{{ pg_admin.username }}"
    login_password: "{{ pg_admin.password }}"

- name: Set user privileges on database
  community.postgresql.postgresql_privs:
    database: "{{ pg_db_name }}"
    roles: "{{ pg_user.username }}"
    privs: ALL
    type: database

    login_host: "{{ pg_host }}"
    port: "{{ pg_port }}"
    login_user: "{{ pg_admin.username }}"
    login_password: "{{ pg_admin.password }}"

I thought I could implement the whole thing via YAML Merge, unfortunately that doesn't seem to work, I always get the following error message:

ERROR! conflicting action statements: login_host, login_user

My attempt to implement it using YAML Merge looks like this:

- &pg_connect {
    login_host: "{{ pg_host }}",
    port: "{{ pg_port }}",
    login_user: "{{ pg_admin.username }}",
    login_password: "{{ pg_admin.password }}"
  }

- name: Create database
  community.postgresql.postgresql_db:
    << : *pg_connect
    name: "{{ pg_db_name }}"


- name: Create database user
  community.postgresql.postgresql_user:
    << : *pg_connect
    name: "{{ pg_user.username }}"
    password: "{{ pg_user.password }}"

- name: Set user privileges on database
  community.postgresql.postgresql_privs:
    << : *pg_connect
    database: "{{ pg_db_name }}"
    roles: "{{ pg_user.username }}"
    privs: ALL
    type: database

Is there a (simple) solution in Ansible to save these redundant lines?


I have now tested with the module_defaults variant for some time, but have encountered the following problem.

I tested with a minimal playbook to avoid side effects as much as possible.

  • pg_connect_modules definition tested as static definition in the vars area of the playbook (like the example) and at runtime via set_fact.
  • pg_connect is defined at runtime via set_fact, in all testing scenarios.
  • The module_defaults definition of task Create database No 1 works without problems.
  • The module_defaults definition of task Create database No 2 always leads to the following error - whereby the playbook execution cannot be started at all, the error appears directly.

    ansible-playbook -i inventory play.yml
    ERROR! The field 'module_defaults' is supposed to be a dictionary or list of dictionaries, the keys of which must be static action, module, or group names. Only the values may contain templates. For example: {'ping': "{{ ping_defaults }}"}

  • I can start the playbook execution at all only if I comment out the task Create database No 2 or at least the module_defaults line of this task.
  • If I have the database tasks in a tasks file and I include them via include_tasks, the include fails; the playbook executes up to that point and then the include fails with the same error.

My playbook looks like this:

play.yml
---

- hosts: testserver
  become: yes

  vars:
    pg_connect_modules:
      community.postgresql.postgresql_db:
        login_host: "localhost"
        port: "5432"
        login_user: "postgres_admin"
        login_password: "myPass"

  tasks:
    - set_fact:
        pg_connect: "{{ common_defaults }}"
      vars:
        common_defaults:
          login_host: "localhost"
          port: "5432"
          login_user: "postgres_admin"
          login_password: "myPass"

    - name: Create database No 1
      community.postgresql.postgresql_db:
        name: "test1"
      module_defaults:
        community.postgresql.postgresql_db: "{{ pg_connect }}"

    - name: Create database No 2
      community.postgresql.postgresql_db:
        name: "test1"
      module_defaults: "{{ pg_connect_modules }}"

Solution

  • Updates:


    A: At any level (play, role, block, task) you can use module_defaults. For example, in a block

    - block:
    
        - name: Create database
          community.postgresql.postgresql_db:
            name: "{{ pg_db_name }}"
    
        - name: Create database user
          community.postgresql.postgresql_user:
            name: "{{ pg_user.username }}"
            password: "{{ pg_user.password }}"
    
        - name: Set user privileges on database
          community.postgresql.postgresql_privs:
            database: "{{ pg_db_name }}"
            roles: "{{ pg_user.username }}"
            privs: ALL
            type: database
    
      module_defaults:
        community.postgresql.postgresql_db:
          login_host: "{{ pg_host }}"
          port: "{{ pg_port }}"
          login_user: "{{ pg_admin.username }}"
          login_password: "{{ pg_admin.password }}"
        community.postgresql.postgresql_user:
          login_host: "{{ pg_host }}"
          port: "{{ pg_port }}"
          login_user: "{{ pg_admin.username }}"
          login_password: "{{ pg_admin.password }}"
        community.postgresql.postgresql_privs:
          login_host: "{{ pg_host }}"
          port: "{{ pg_port }}"
          login_user: "{{ pg_admin.username }}"
          login_password: "{{ pg_admin.password }}"
    

    It's practical to combine and declare the dictionary module_defaults outside the code. For example, in the group_vars

    shell> cat group_vars/all/my_defaults.yml
    my_module_defaults_list:
      - modules:
          - community.postgresql.postgresql_db
          - community.postgresql.postgresql_user
          - community.postgresql.postgresql_privs
        defaults:
          login_host: "{{ pg_host }}"
          port: "{{ pg_port }}"
          login_user: "{{ pg_admin.username }}"
          login_password: "{{ pg_admin.password }}"
    
    my_module_defaults_str: |
      {% for i in my_module_defaults_list %}
      {{ dict(i.modules|product([i.defaults])) }}
      {% endfor %}
    my_module_defaults: "{{ my_module_defaults_str|from_yaml }}"
    

    given the variables below

      pg_host: pb.example.com
      pg_port: 5432
      pg_admin:
        username: admin
        password: admin_passwd
    

    The dictionary gives

      my_module_defaults:
        community.postgresql.postgresql_db:
          login_host: pb.example.com
          login_password: admin_passwd
          login_user: admin
          port: 5432
        community.postgresql.postgresql_privs:
          login_host: pb.example.com
          login_password: admin_passwd
          login_user: admin
          port: 5432
        community.postgresql.postgresql_user:
          login_host: pb.example.com
          login_password: admin_passwd
          login_user: admin
          port: 5432
    

    Unfortunately, it's not possible to use it directly in a play. For some reason, the keys of the dictionary must be static

    - block:
    
        - name: Create database
          community.postgresql.postgresql_db:
            name: "{{ pg_db_name }}"
    
        - name: Create database user
          community.postgresql.postgresql_user:
            name: "{{ pg_user.username }}"
            password: "{{ pg_user.password }}"
    
        - name: Set user privileges on database
          community.postgresql.postgresql_privs:
            database: "{{ pg_db_name }}"
            roles: "{{ pg_user.username }}"
            privs: ALL
            type: database
    
      module_defaults: "{{ my_module_defaults }}"
    

    The above code will cause the error:

    ERROR! The field 'module_defaults' is supposed to be a dictionary or list of dictionaries, the keys of which must be static action, module, or group names. Only the values may contain templates. For example: {'ping': "{{ ping_defaults }}"}

    As a workaround, you can create the below template

    shell> cat postgresql.yml.j2
    - block:
    
        - name: Create database
          community.postgresql.postgresql_db:
            name: "{{ pg_db_name }}"
    
        - name: Create database user
          community.postgresql.postgresql_user:
            name: "{{ pg_user.username }}"
            password: "{{ pg_user.password }}"
    
        - name: Set user privileges on database
          community.postgresql.postgresql_privs:
            database: "{{ pg_db_name }}"
            roles: "{{ pg_user.username }}"
            privs: ALL
            type: database
    
      module_defaults:
    {{ my_module_defaults|to_nice_yaml(indent=2)|indent(4, first=true) }}
    

    and include the created file in the play

        - template:
            src: postgresql.yml.j2
            dest: "{{ playbook_dir }}/postgresql.yml"
          delegate_to: localhost
          run_once: true
          tags: build
        - include_tasks: postgresql.yml
    

    shell> cat postgresql.yml
    - block:
    
        - name: Create database
          community.postgresql.postgresql_db:
            name: "db_01"
    
        - name: Create database user
          community.postgresql.postgresql_user:
            name: "user"
            password: "user_passwd"
    
        - name: Set user privileges on database
          community.postgresql.postgresql_privs:
            database: "db_01"
            roles: "user"
            privs: ALL
            type: database
    
      module_defaults:
        community.postgresql.postgresql_db:
          login_host: pb.example.com
          login_password: admin_passwd
          login_user: admin
          port: 5432
        community.postgresql.postgresql_privs:
          login_host: pb.example.com
          login_password: admin_passwd
          login_user: admin
          port: 5432
        community.postgresql.postgresql_user:
          login_host: pb.example.com
          login_password: admin_passwd
          login_user: admin
          port: 5432
    

    The proper solution is to create Module defaults group. There is no group declared in the source atm. You can create it. For example, find out the path to your collections and create the group

    shell> cat ~/.local/lib/python3.9/site-packages/ansible_collections/community/postgresql/meta/runtime.yml 
    ---
    requires_ansible: '>=2.9.10'
    action_groups:
      postgresql:
        - postgresql_db
        - postgresql_privs
        - postgresql_user
    

    Then the block below should work as expected

        - block:
    
            - name: Create database
              community.postgresql.postgresql_db:
                name: "db_01"
    
            - name: Create database user
              community.postgresql.postgresql_user:
                name: "user"
                password: "user_passwd"
    
            - name: Set user privileges on database
              community.postgresql.postgresql_privs:
                database: "db_01"
                roles: "user"
                privs: ALL
                type: database
    
          module_defaults:
            group/community.postgresql.postgresql:
              login_host: pb.example.com
              login_password: admin_passwd
              login_user: admin
              port: 5432
    

    You can open a request if you want to. It will be necessary to review the consistency of all parameters of all postgresql modules and create the group(s).


    Notes:

    • In a mission-critical environment provide the login_password in a limited scope. See Ansible passing vault password file to playbook.

    • If you want to use module_defaults at a play level the dictionary must be available in this scope. See Ansible, module_defaults and package module (obsolete; broken in 2.12).

    • Quoting from CHANGELOG-v2.12.rst Breaking Changes: "Action, module, and group names in module_defaults must be static values. Their values can still be templates."

    Example of a complete playbook for testing the error and the workaround

    shell> cat pb3.yml
    - hosts: localhost
    
      vars:
    
        my_module_defaults:
          debug:
            msg: common default
    
        debug_defaults:
          msg: common default
    
      tasks:
    
        - name: Work as expected
          debug:
          module_defaults:
            debug: "{{ debug_defaults }}"
          
    #    - name: ERROR
    #      debug:
    #      module_defaults: "{{ my_module_defaults }}"
    #
    # ERROR! The field 'module_defaults' is supposed to be a dictionary or
    # list of dictionaries, the keys of which must be static action, module,
    # or group names. Only the values may contain templates. For example:
    # {'ping': "{{ ping_defaults }}"}
    
        - name: Workaround
          template:
            src: my_module.yml.j2
            dest: "{{ playbook_dir }}/my_module.yml"
        - include_tasks: my_module.yml
    
    shell> cat my_module.yml.j2
    - block:
    
        - debug:
    
      module_defaults:
    {{ my_module_defaults|to_nice_yaml(indent=2)|indent(4, first=true) }}
    
    shell> cat my_module.yml
    - block:
    
        - debug:
    
      module_defaults:
        debug:
          msg: common default