TL;DR: I need to flatten an Ansible dictionary of dictionaries so that sub items are moved at the root level and their original parent key added to them.
Context: I need to populate a TOML file using Ansible with configuration options coming from an Ansible variable/fact. I am using community.general.ini_file to populate the .toml file because it works fine (in my case).
My input variable/fact is simple:
config:
param0_1: value0_1 # This key-value pair has no section/table
section1: # This dict contains all the key-value pairs for section/table "section1"
param1_1: value1_1
section2: # Section/table "section2"
param2_1: value2_1
param2_2: value2_2
And I want the final TOML file to looks like this (= have those settings):
param0_1 = 'value0_1'
[section1]
param1_1 = 'value1_1'
[section2]
param2_1 = 'value2_1'
param2_2 = 'value2_2'
I managed to obtain what I want using this Server Fault "Using Ansible ini_file with nested dict?" answer which uses Jinja and JSON notation to process/transform the input in the desired output.
Here is a playbook with such solution:
# Usage: ansible-playbook playbook.yml -C -D
---
- hosts: 127.0.0.1
connection: local
gather_facts: false
vars:
config:
param0_1: value0_1
section1:
param1_1: value1_1
section2:
param2_1: value2_1
param2_2: value2_2
config_flat_via_jinja_json: >-
[
{% for section, subdict in config.items() %}
{% if subdict is mapping %}
{% for key, value in subdict.items() %}
{
"section": "{{ section }}",
"key": "{{ key }}",
"value": "{{ value }}"
}
{% if not loop.last %}
,
{% endif %}
{% endfor %}
{% else %}
{
"section": null,
"key": "{{ section }}",
"value": "{{ subdict }}"
}
{% endif %}
{% if not loop.last %}
,
{% endif %}
{% endfor %}
]
tasks:
- set_fact:
config:
param0_1: value0_1
section1:
param1_1: value1_1
section2:
param2_1: value2_1
param2_2: value2_2
- set_fact:
config_flat_via_jinja_json: >-
[
{% for section, subdict in config.items() %}
{% if subdict is mapping %}
{% for key, value in subdict.items() %}
{
"section": "{{ section }}",
"key": "{{ key }}",
"value": "{{ value }}"
}
{% if not loop.last %}
,
{% endif %}
{% endfor %}
{% else %}
{
"section": null,
"key": "{{ section }}",
"value": "{{ subdict }}"
}
{% endif %}
{% if not loop.last %}
,
{% endif %}
{% endfor %}
]
- debug:
var: config
- debug:
var: config_flat_via_jinja_json
- name: loop on config_flat_via_jinja_json | list
debug:
var: item
loop: "{{ config_flat_via_jinja_json | list }}"
- name: Configure application
# Using ini_file module to populates the TOML file. It works for our need, ini sections are like TOML tables.
ini_file:
path: config.toml
section: '{{ item.section }}'
option: '{{ item.key }}'
value: '{{ item.value }}'
# Will remove configuration line from config file if config value is empty
state: '{{ "present" if item.value != "" else "absent" }}'
loop: "{{ config_flat_via_jinja_json | list }}"
Which has the following output for "config_flat_via_jinja_json":
TASK [debug] ******************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"config_flat_via_jinja_json": [
{
"key": "param0_1",
"section": null,
"value": "value0_1"
},
{
"key": "param1_1",
"section": "section1",
"value": "value1_1"
},
{
"key": "param2_1",
"section": "section2",
"value": "value2_1"
},
{
"key": "param2_2",
"section": "section2",
"value": "value2_2"
}
]
}
I works but it looks like an hack for something I think is doable in native Ansible.
Is it?
Note 1: Because the TOML file gets altered from outside of Ansible I cannot use an Ansible template here. Beside I think it's irrelevant to the main issue I'm struggling with.
Note 2: I am using Ansible v2.10 (let's hope such "old" version is not a disqualifier for solutions).
We can simplify the logic required to transform the config dictionary into a more useful data structure. Our goal here is to go from a single dictionary into a list of dictionaries formatted such that we can use it with the subelements filter.
Here's one possible solution:
- hosts: localhost
gather_facts: false
vars:
config:
param0_1: value0_1
section1:
param1_1: value1_1
section2:
param2_1: value2_1
param2_2: value2_2
tasks:
- loop: "{{ config.keys() }}"
vars:
_config: []
set_fact:
_config: >-
{% if config[item] is mapping -%}
{{ _config + [{'name': item, 'items': config[item]|dict2items}] }}
{% else -%}
{{ _config + [{'name': 'default', 'items': {item: config[item]} | dict2items}] }}
{% endif -%}
- debug:
var: _config
- name: Configure application
ini_file:
path: config.toml
section: '{{ (item.0.name == "default") | ternary("", item.0.name) }}'
option: '{{ item.1.key }}'
value: '{{ item.1.value }}'
state: '{{ "present" if item.1.value != "" else "absent" }}'
loop: "{{ _config | subelements('items') }}"
The set_fact task produces a data structure that looks like this:
[
{
"items": [
{
"key": "param0_1",
"value": "value0_1"
}
],
"name": "default"
},
{
"items": [
{
"key": "param1_1",
"value": "value1_1"
}
],
"name": "section1"
},
{
"items": [
{
"key": "param2_1",
"value": "value2_1"
},
{
"key": "param2_2",
"value": "value2_2"
}
],
"name": "section2"
}
]
Running the above playbook produces a config.toml that looks like:
param0_1 = value0_1
[section1]
param1_1 = value1_1
[section2]
param2_1 = value2_1
param2_2 = value2_2
I have tested this with Ansible 2.10 on Python 3.10 and it seems to work:
ansible-playbook 2.10.17
config file = /src/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.10/site-packages/ansible
executable location = /usr/local/bin/ansible-playbook
python version = 3.10.14 (main, Sep 4 2024, 06:02:46) [GCC 12.2.0]