Search code examples
ansibledevopsufw

Ansible UFW: parallel loop to make execution faster

I am making an Ansible task that adds UFW rules. There are sometimes a lot of rules to be added and they are added sequentially (which is expected behaviour of course).

But I don't care about the order of the rules as they don't overlap (they are more like whitelisting from a deny in, deny out, deny forward defaults).

So I thought about using the async/poll features of Ansible to make the loop execution async, as in this stackexchange devops thread.

But is seems using such async does not work for loops, or at least for my loop. If I set poll to 0, the rules are added sequentially as before and not faster. If I set poll to a positive number, it runs even slower than before.

A code snippet of the rule-adding code:

- name: configure | rules
  ufw:
    rule: "{{ item.rule }}"
    interface: "{{ item.interface | default('') }}"
    direction: "{{ item.direction | default('in') }}"
    from_ip: "{{ item.from_ip | default('any') }}"
    to_ip: "{{ item.to_ip | default('any') }}"
    from_port: "{{ item.from_port | default('') }}"
    to_port: "{{ item.to_port | default('') }}"
    protocol: "{{ item.protocol | default('any') }}"
    route: "{{ item.route | default(omit) }}"
    log: "{{ item.log | default(false) }}"
    comment: "{{ item.comment | default(omit) }}"
  with_items: "{{ ufw_rules }}"
  register: _create_instances
  async: 1000
  poll: 0
  notify: reload ufw
  tags:
    - ufw-configure-rules


- name: Wait for creation to finish
  async_status:
    jid: "{{ item.ansible_job_id }}"
  register: _jobs
  until: _jobs.finished
  delay: 5  # Check every 5 seconds. Adjust as you like.                                                                                                                                                    
  retries: 10  # Retry up to 10 times. Adjust as needed.                                                                                                                                                    
  with_items: "{{ _create_instances.results }}"

The result is my loop runs item by item, not faster.

About my configuration:

  1. Ansible is configured for pipelining and connects to remote hosts through ssh multiplexing.
  2. Ansible --version 2.5.1

I could generate an ufw rules files or iptables, but my goal in the beginning is interacting with UFW through ansible API.

Solution

  • Consider dividing tasks into smaller groups. For example:

    - name: configure | rules
  ufw:
  .
  .
  .
with_items: "{{ ufw_rules_set1 }}"
.
.
.

- name: configure next | rules
  ufw:
  .
  .
  .
with_items: "{{ ufw_rules_set2 }}"
.
.
.

    Then async is going to work and you encounter about 2x speedup. Of course the more stages the more speedup but at some point you reach maximum and dividing further will make you worse off.