Search code examples
ansiblebatchsize

Controlling Ansible playbook execution

Is it possible to control batch execution in percentage, but applying it individually to groups of hosts in the Ansible playbook?

my hosts file:

group_1 - has 5 hosts
group_2 - has 18 hosts
group_3 - has 4 hosts

And my playbook like this:

- name: "Apply percent per group"
  hosts: all
  serial: "30%"

Currently, when I run like this, Ansible concatenate all groups and applies the 30% serial, so it runs 8 hosts per round and linearly, so it ends up getting all the hosts in group_1 at once.

Is there a way to apply this 30% per group individually?

So per batch it would run 1 host from group_1, 5 hosts from group_2 and 1 host from group_3.

It would even give me the possibility to increase the percentage that would still not cause unavailability and decrease execution time.

Solution

  • Create the batch on your own. For example, given the inventory

    shell> cat hosts
[group_1]
host_1_[01:05]
[group_2]
host_2_[01:18]
[group_3]
host_3_[01:04]

    and the variable my_serial_pc

      my_serial_pc: 30
    1. Declare the variables
      count_1: "{{ groups.group_1|length * my_serial_pc|int // 100 }}"
  count_2: "{{ groups.group_2|length * my_serial_pc|int // 100 }}"
  count_3: "{{ groups.group_3|length * my_serial_pc|int // 100 }}"
  batch_1: "{{ groups.group_1|batch(count_1|int) }}"
  batch_2: "{{ groups.group_2|batch(count_2|int) }}"
  batch_3: "{{ groups.group_3|batch(count_3|int) }}"
  my_serial: "{{ count_1|int + count_2|int + count_3|int }}"

    give

      count_1: 1
  count_2: 5
  count_3: 1
  batch_1:
    - [host_1_01]
    - [host_1_02]
    - [host_1_03]
    - [host_1_04]
    - [host_1_05]
  
  batch_2:
    - [host_2_01, host_2_02, host_2_03, host_2_04, host_2_05]
    - [host_2_06, host_2_07, host_2_08, host_2_09, host_2_10]
    - [host_2_11, host_2_12, host_2_13, host_2_14, host_2_15]
    - [host_2_16, host_2_17, host_2_18]
 
  batch_3:
    - [host_3_01]
    - [host_3_02]
    - [host_3_03]
    - [host_3_04]

  my_serial: 7
    1. zip and flatten the batches
      my_hosts: "{{ batch_1|
                zip_longest(batch_2)|map('flatten')|
                zip_longest(batch_3)|map('flatten')|flatten }}"
    1. Create a new group my_group
            - add_host:
            name: "{{ item }}"
            groups: my_group
            my_serial: "{{ my_serial }}"
          loop: "{{ my_hosts }}"
    1. Run the next play with the group my_group and serial my_serial
    - hosts: my_group
  serial: "{{ hostvars[groups.my_group.0].my_serial }}"
  tasks:
    - debug:
        var: ansible_play_batch|to_yaml
      run_once: true

    gives

    PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_01] => 
  ansible_play_batch|to_yaml: |-
    [host_1_01, host_2_01, host_2_02, host_2_03, host_2_04, host_2_05, host_3_01]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_02] => 
  ansible_play_batch|to_yaml: |-
    [host_1_02, host_2_06, host_2_07, host_2_08, host_2_09, host_2_10, host_3_02]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_03] => 
  ansible_play_batch|to_yaml: |-
    [host_1_03, host_2_11, host_2_12, host_2_13, host_2_14, host_2_15, host_3_03]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_04] => 
  ansible_play_batch|to_yaml: |-
    [host_1_04, host_2_16, host_2_17, host_2_18, host_3_04, host_1_05]

    Example of a complete playbook for testing

    shell> cat pb.yml
- hosts: all

  vars:

    my_serial_pc: 30
    count_1: "{{ groups.group_1|length * my_serial_pc|int // 100 }}"
    count_2: "{{ groups.group_2|length * my_serial_pc|int // 100 }}"
    count_3: "{{ groups.group_3|length * my_serial_pc|int // 100 }}"
    batch_1: "{{ groups.group_1|batch(count_1|int) }}"
    batch_2: "{{ groups.group_2|batch(count_2|int) }}"
    batch_3: "{{ groups.group_3|batch(count_3|int) }}"
    my_hosts: "{{ batch_1|
                  zip_longest(batch_2)|map('flatten')|
                  zip_longest(batch_3)|map('flatten')|flatten }}"
    my_serial: "{{ count_1|int + count_2|int + count_3|int }}"

  tasks:

    - block:
        - debug:
            msg: |
              count_1: {{ count_1 }}
              count_2: {{ count_2 }}
              count_3: {{ count_3 }}
              batch_1:
                {{ batch_1|to_yaml|indent(2) }}
              batch_2:
                {{ batch_2|to_yaml|indent(2) }}
              batch_3:
                {{ batch_3|to_yaml|indent(2) }}
              my_serial: {{ my_serial }}
          when: debug|d(false)|bool
        - add_host:
            name: "{{ item }}"
            groups: my_group
            my_serial: "{{ my_serial }}"
          loop: "{{ my_hosts }}"
      run_once: true

- hosts: my_group
  serial: "{{ hostvars[groups.my_group.0].my_serial }}"
  tasks:
    - debug:
        var: ansible_play_batch|to_yaml
      run_once: true

    gives

    shell> ansible-playbook pb.yml -e debug=true

PLAY [all] ************************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_01] => 
  msg: |-
    count_1: 1
    count_2: 5
    count_3: 1
    batch_1:
      - [host_1_01]
      - [host_1_02]
      - [host_1_03]
      - [host_1_04]
      - [host_1_05]
  
    batch_2:
      - [host_2_01, host_2_02, host_2_03, host_2_04, host_2_05]
      - [host_2_06, host_2_07, host_2_08, host_2_09, host_2_10]
      - [host_2_11, host_2_12, host_2_13, host_2_14, host_2_15]
      - [host_2_16, host_2_17, host_2_18]
  
    batch_3:
      - [host_3_01]
      - [host_3_02]
      - [host_3_03]
      - [host_3_04]
  
    my_serial: 7

TASK [add_host] *******************************************************************************************
changed: [host_1_01] => (item=host_1_01)
changed: [host_1_01] => (item=host_2_01)
changed: [host_1_01] => (item=host_2_02)
changed: [host_1_01] => (item=host_2_03)
changed: [host_1_01] => (item=host_2_04)
changed: [host_1_01] => (item=host_2_05)
changed: [host_1_01] => (item=host_3_01)
changed: [host_1_01] => (item=host_1_02)
changed: [host_1_01] => (item=host_2_06)
changed: [host_1_01] => (item=host_2_07)
changed: [host_1_01] => (item=host_2_08)
changed: [host_1_01] => (item=host_2_09)
changed: [host_1_01] => (item=host_2_10)
changed: [host_1_01] => (item=host_3_02)
changed: [host_1_01] => (item=host_1_03)
changed: [host_1_01] => (item=host_2_11)
changed: [host_1_01] => (item=host_2_12)
changed: [host_1_01] => (item=host_2_13)
changed: [host_1_01] => (item=host_2_14)
changed: [host_1_01] => (item=host_2_15)
changed: [host_1_01] => (item=host_3_03)
changed: [host_1_01] => (item=host_1_04)
changed: [host_1_01] => (item=host_2_16)
changed: [host_1_01] => (item=host_2_17)
changed: [host_1_01] => (item=host_2_18)
changed: [host_1_01] => (item=host_3_04)
changed: [host_1_01] => (item=host_1_05)

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_01] => 
  ansible_play_batch|to_yaml: |-
    [host_1_01, host_2_01, host_2_02, host_2_03, host_2_04, host_2_05, host_3_01]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_02] => 
  ansible_play_batch|to_yaml: |-
    [host_1_02, host_2_06, host_2_07, host_2_08, host_2_09, host_2_10, host_3_02]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_03] => 
  ansible_play_batch|to_yaml: |-
    [host_1_03, host_2_11, host_2_12, host_2_13, host_2_14, host_2_15, host_3_03]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_04] => 
  ansible_play_batch|to_yaml: |-
    [host_1_04, host_2_16, host_2_17, host_2_18, host_3_04, host_1_05]

PLAY RECAP ************************************************************************************************
host_1_01: ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host_1_02: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host_1_03: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host_1_04: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0