Search code examples
ansibleansible-inventory

Ansible dynamic inventory script

I'm trying to upgrade various servers across multiple environments. Upgrades need to be run sequentially. It would be better if i could use ansible dynamic inventories for this as some of the environments can contain 100's of servers.

The plan is to have the first Ansible Play discover the inventory items and, create the dynamic inventory for the particular environment, a second Ansible Play would then make use of the dynamic inventory and perform the required actions on these devices

Taking a small environment as an example, i've done some work and the final task of the first play is

- name: Create dynamic inventory by combining facts from previous tasks
  add_host:
    name: "{{ item.fqdn }}"
    groups: targets
  with_items: "{{ server_details }}"
  when: item.id == another.id
  no_log: true

- name: Print the dynamic inventory
  debug:
    var: groups.targets

That gives me the following

ok: [mgmt-vip.us.acme.com] => {
    "groups.targets": [
        "server01.us.acme.com",
        "server03.us.acme.com",
        "server04.us.acme.com",
        "server02.us.acme.com"
    ]
}

A second Play should then be able to use the above dynamic inventory

- name: Sequential tasks
  hosts: targets
  gather_facts: false
  connection: local
  serial: 1

  tasks:
    - name: Print the dynamic inventory
      debug:
        msg: "{{item}}"
      with_items: "{{targets}}"

    - name: Ping Server if it is part of the dynamic inventory
      import_role:
        name: ping

When i run the playbook, the Print the dynamic inventory task executes successfully but, the Sequential Tasks play is skipped with the message "skipping: no hosts matched".

I can't proceed with the actual upgrade tasks until i can at least access/ping the servers in the dynamic inventory

Solution

  • I identified the issue today when doing additional testing. To run my playbook, which contains two plays, i was calling ansible-playbook playbookname.yml -l some_servername it was the limit when calling the playbook that caused the second inventory (the dynamic one) to error.

    To Reproduce the error: sample.json contains the list of Servers that was dynamically obtained

    {
    "servers": [
        "server01.us.acme.com",
        "server03.us.acme.com",
        "server04.us.acme.com",
        "server02.us.acme.com"
    ]
}

    inventory is a list of my Management Servers, one for each environment and 100's of environments (reduced for demo)

    [p1]
mgmt_server_01.us.acme.com

[p1:vars]
password = "{{ authentication_data.p1_passphrase }}"

[p2]
mgmt_server_02.apac.acme.com

[p2:vars]
password = "{{ authentication_data.p2_passphrase }}"

[p3]
mgmt_server_03.emea.acme.com

[p3:vars]
password = "{{ authentication_data.p3_passphrase }}"

[mgmt_servers:children]
p1
p2
p3

[mgmt_servers:vars]
username = "admin"

    dynamic_inv.yml is a playbook containing two plays.

    • First play reads in the sample.json file and creates a dynamic inventory called sample_group
    • Second play processes tasks sequentially, one Server at a time
    ---
- hosts: mgmt_servers
  name: Read sample of servers from json and create dynamic inventory. Tasks happen for ALL inventory Servers
  gather_facts: false
  connection: local
  vars:
    json_var: "{{ lookup('file', './sample.json') | from_json }}"

  tasks:
    - name: print contents of json_var
      debug:
        var: json_var 

    - name: Read json to get sample servers 
      debug:
        msg: "Server names are {{item}}"
      with_items: "{{json_var.servers}}"

    - name: Create dynamic inventory from json_vars
      add_host:
        name: "{{ item }}"
        groups: sample_group
      with_items: "{{ json_var.servers }}"
      no_log: false

    - name: Print the dynamic inventory
      debug:
        var: groups.sample_group

- hosts: sample_group
  name: tasks iterate sequentially, one server at a time
  gather_facts: false
  connection: local
  order: sorted
  serial: 1

  tasks:
    - name: Print something to show the flow of the playbook
      debug: msg="This line is printed in the beginning for activity on each sample server in the inventory"
    
    - name: Print the Server name
      debug:
        msg: "The Server name is {{inventory_hostname}}"
    
    - name: Print another message
      debug: msg="All 3x tasks completed for this inventory item, moving to the next inventory item"

    REPRODUCING THE ISSUE: ansible-playbook testing/dynamic_inv.yml -i testing/inventory -l mgmt_server_02.us.acme.com produces

    TASK [Print the dynamic inventory] ********************************************************************************************************************************************************************************
ok: [mgmt_server_02.us.acme.com] => {
    "groups.sample_group": [
        "server01.us.acme.com",
        "server03.us.acme.com",
        "server04.us.acme.com",
        "server02.us.acme.com"
    ]
}

PLAY [tasks iterate sequentially, one server at a time] ***********************************************************************************************************************************************************
skipping: no hosts matched

PLAY RECAP *********

    i.e. The first play executes but the second fails.

    PROBLEM: The first inventory file contains 100's of servers and i don't want to create a dynamic inventory for each one so, I must limit somehow.

    SOLUTION: Change the initial hosts from being static to being a variable and pass an extra var when calling the playbook.

    ---
- hosts: "{{ upgrade }}"
  name: Read sample of servers from json and create dynamic inventory. Tasks happen for ALL inventory Servers
  gather_facts: false
  connection: local
  vars:
    json_var: "{{ lookup('file', './sample.json') | from_json }}"

  tasks:
    - name: print contents of json_var
      debug:
        var: json_var 

    - name: Read json to get sample servers 
      debug:
        msg: "Server names are {{item}}"
      with_items: "{{json_var.servers}}"

    - name: Create dynamic inventory from json_vars
      add_host:
        name: "{{ item }}"
        groups: sample_group
      with_items: "{{ json_var.servers }}"
      no_log: false

    - name: Print the dynamic inventory
      debug:
        var: groups.sample_group

- hosts: sample_group
  name: tasks iterate sequentially, one server at a time
  gather_facts: false
  connection: local
  order: sorted
  serial: 1

  tasks:
    - name: Print something to show the flow of the playbook
      debug: msg="This line is printed in the beginning for activity on each sample server in the inventory"
    
    - name: Print the Server name
      debug:
        msg: "The Server name is {{inventory_hostname}}"
    
    - name: Print another message
      debug: msg="All 3x tasks completed for this inventory item, moving to the next inventory item"

    call the above with ansible-playbook testing/dynamic_inv.yml -i testing/inventory -e upgrade=mgmt_server_02.apac.acme.com

    produces ... ...

    TASK [Print the dynamic inventory] ********************************************************************************************************************************************************************************
ok: [mgmt_server_02.apac.acme.com] => {
    "groups.sample_group": [
        "server01.us.acme.com",
        "server03.us.acme.com",
        "server04.us.acme.com",
        "server02.us.acme.com"
    ]
}

PLAY [tasks iterate sequentially, one server at a time] ***********************************************************************************************************************************************************

TASK [Print something to show the flow of the playbook] ***********************************************************************************************************************************************************
ok: [server01.us.acme.com] => {
    "msg": "This line is printed in the beginning for activity on each sample server in the inventory"
}

TASK [Print the Server name] **************************************************************************************************************************************************************************************
ok: [server01.us.acme.com] => {
    "msg": "The Server name is server01.us.acme.com"
}

TASK [Print another message] **************************************************************************************************************************************************************************************
ok: [server01.us.acme.com] => {
    "msg": "All 3x tasks completed for this inventory item, moving to the next inventory item"
}

PLAY [tasks iterate sequentially, one server at a time] ***********************************************************************************************************************************************************

TASK [Print something to show the flow of the playbook] ***********************************************************************************************************************************************************
ok: [server02.us.acme.com] => {
    "msg": "This line is printed in the beginning for activity on each sample server in the inventory"
}

TASK [Print the Server name] **************************************************************************************************************************************************************************************
ok: [server02.us.acme.com] => {
    "msg": "The Server name is server02.us.acme.com"
}

TASK [Print another message] **************************************************************************************************************************************************************************************
ok: [server02.us.acme.com] => {
    "msg": "All 3x tasks completed for this inventory item, moving to the next inventory item"
}

PLAY [tasks iterate sequentially, one server at a time] ***********************************************************************************************************************************************************

TASK [Print something to show the flow of the playbook] ***********************************************************************************************************************************************************
ok: [server03.us.acme.com] => {
    "msg": "This line is printed in the beginning for activity on each sample server in the inventory"

    ... ...