How to combine items from a Dictionary into a list of dictionaries based on matching key/value details

I have a list of dictionaries

    "tor_vlans": [
        {
            "switch-ls01": {
                "Vlan1": {
                    "id": "1",
                    "ip": "unassigned",
                    "ok": "NO",
                    "protocol": "down",
                    "status": "up"
                },
                "Vlan10": {
                    "id": "10",
                    "ip": "10.10.10.2/24",
                    "ok": "YES",
                    "protocol": "up",
                    "status": "up"
                },
                "Vlan20": {
                    "id": "20",
                    "ip": "10.10.20.2/24",
                    "ok": "YES",
                    "protocol": "up",
                    "status": "up"
                }
            }
        }
    ]

Using the ansible.utils.ipaddr module I've extracted the ip values from the above and created a new dictionary containing additional details for each subnet

    "subnet_details": {
        "10.10.10.2/24": {
            "dhcp_scope": "10.10.10.4-10.10.10.254",
            "tor_1_ip": "10.10.10.2/24",
            "tor_2_ip": "10.10.10.3/24",
            "vrrp": "10.10.10.1/24"
        },
        "10.10.20.2/24": {
            "dhcp_scope": "10.10.20.4-10.10.20.254",
            "tor_1_ip": "10.10.20.2/24",
            "tor_2_ip": "10.10.20.3/24",
            "vrrp": "10.10.20.1/24"
        }
    }

I'm attempting to combine both sets of dictionaries into a new list of dictionaries by matching the ip value from the tor_vlans list with the keys from subnet_details

Using Jinja2 templating below

    - name: Combine both based on matching ips and keys
      ansible.builtin.set_fact:
        my_new_dict: |
          {% for tor in tor_vlans %}
          {% for key,value in tor.items() %}
          {% for vlan,facts in value.items() %}
          {% for cidr,param in subnet_details.items() %}
          {% if cidr in facts %}
          {{ key }}:
            {{ vlan }}:
              {{ facts }}
              {{ param }}
          {% else %}
          {{ key }}:
            {{ vlan }}:
              {{ facts }}
          {% endif %}
          {% endfor %}
          {% endfor %}
          {% endfor %}
          {% endfor %}

The expected output would be

    "my_new_dict": [
        {
            "switch-ls01": {
                "Vlan1": {
                    "id": "1",
                    "ip": "unassigned",
                    "ok": "NO",
                    "protocol": "down",
                    "status": "up"
                },
                "Vlan10": {
                    "id": "10",
                    "ip": "10.10.10.2/24",
                    "ok": "YES",
                    "protocol": "up",
                    "status": "up",
                    "dhcp_scope": "10.10.10.4-10.10.10.254",
                    "tor_1_tep_ip": "10.10.10.2/24",
                    "tor_2_tep_ip": "10.10.10.3/24",
                    "vrrp": "10.10.10.1/24"
                },
                "Vlan20": {
                    "id": "20",
                    "ip": "10.10.20.2/24",
                    "ok": "YES",
                    "protocol": "up",
                    "status": "up",
                    "dhcp_scope": "10.10.20.4-10.10.20.254",
                    "tor_1_tep_ip": "10.10.20.2/24",
                    "tor_2_tep_ip": "10.10.20.3/24",
                    "vrrp": "10.10.20.1/24"
                }
            }
        }
    ]

The actual output that i'm getting is

    "my_new_dict|from_yaml": {
        "switch-ls01": {
            "Vlan20": {
                "id": "20",
                "ip": "10.10.20.2/24",
                "ok": "YES",
                "protocol": "up",
                "status": "up"
            }
        }
    }

If there's a way of combing other than using Jinja, please let me know.

Likewise, if the subnet details could be created directly in tor_vlans using the ansible.utils.ipaddr module, I can't figure it out.

The "Vlan1", "Vlan10", etc... that are causing me issues when trying to use the ipaddr module directly on tor_vlans as these keys are dynamically learnt so i can't presume they will always be the same, i.e. sometimes they could be "Vlan25", "Vlan55", etc...

I've found a good few posts on this for Python but, can't seem to find any similar Ansible posts

Solution

Create list of updates

    update: |
      {% filter from_yaml %}
      {% for vlan in tor_vlans %}
      {% for sw,vlans in vlan.items() %}
      [ {{ sw }}: {
      {% for k,v in vlans.items() %}
      {% for s,d in subnet_details.items() %}
      {% if [v.ip]|ansible.utils.ipaddr(s)|length > 0 %}
      {{ k }}: {{ v|combine(d) }},
      {% endif %}
      {% endfor %}
      {% endfor %} },
      {% endfor %} ]
      {% endfor %}
      {% endfilter %}

gives

  update:
  - switch-ls01:
      Vlan10:
        dhcp_scope: 10.10.10.4-10.10.10.254
        id: '10'
        ip: 10.10.10.2/24
        ok: 'YES'
        protocol: up
        status: up
        tor_1_ip: 10.10.10.2/24
        tor_2_ip: 10.10.10.3/24
        vrrp: 10.10.10.1/24
      Vlan20:
        dhcp_scope: 10.10.20.4-10.10.20.254
        id: '20'
        ip: 10.10.20.2/24
        ok: 'YES'
        protocol: up
        status: up
        tor_1_ip: 10.10.20.2/24
        tor_2_ip: 10.10.20.3/24
        vrrp: 10.10.20.1/24

zip the lists and combine the items

    my_new_dict: "{{ tor_vlans |
                     zip(update) |
                     map('combine', recursive=True) }}"

gives

  my_new_dict:
  - switch-ls01:
      Vlan1:
        id: '1'
        ip: unassigned
        ok: 'NO'
        protocol: down
        status: up
      Vlan10:
        dhcp_scope: 10.10.10.4-10.10.10.254
        id: '10'
        ip: 10.10.10.2/24
        ok: 'YES'
        protocol: up
        status: up
        tor_1_ip: 10.10.10.2/24
        tor_2_ip: 10.10.10.3/24
        vrrp: 10.10.10.1/24
      Vlan20:
        dhcp_scope: 10.10.20.4-10.10.20.254
        id: '20'
        ip: 10.10.20.2/24
        ok: 'YES'
        protocol: up
        status: up
        tor_1_ip: 10.10.20.2/24
        tor_2_ip: 10.10.20.3/24
        vrrp: 10.10.20.1/24

^{Example of a complete playbook for testing
- hosts: all

vars:

tor_vlans:
- switch-ls01:
Vlan1:
id: '1'
ip: unassigned
ok: 'NO'
protocol: down
status: up
Vlan10:
id: '10'
ip: 10.10.10.2/24
ok: 'YES'
protocol: up
status: up
Vlan20:
id: '20'
ip: 10.10.20.2/24
ok: 'YES'
protocol: up
status: up

subnet_details:
10.10.10.2/24:
dhcp_scope: 10.10.10.4-10.10.10.254
tor_1_ip: 10.10.10.2/24
tor_2_ip: 10.10.10.3/24
vrrp: 10.10.10.1/24
10.10.20.2/24:
dhcp_scope: 10.10.20.4-10.10.20.254
tor_1_ip: 10.10.20.2/24
tor_2_ip: 10.10.20.3/24
vrrp: 10.10.20.1/24

update: |
{% filter from_yaml %}
{% for vlan in tor_vlans %}
{% for sw,vlans in vlan.items() %}
[ {{ sw }}: {
{% for k,v in vlans.items() %}
{% for s,d in subnet_details.items() %}
{% if [v.ip]|ansible.utils.ipaddr(s)|length > 0 %}
{{ k }}: {{ v|combine(d) }},
{% endif %}
{% endfor %}
{% endfor %} },
{% endfor %} ]
{% endfor %}
{% endfilter %}

my_new_dict: "{{ tor_vlans |
zip(update) |
map('combine', recursive=True) }}"

tasks:

- debug:
var: update
- debug:
var: my_new_dict}