Modify each value of a dictionary to contain a child value

Question

Let's say I have a dictionary called packages in an Ansible playbook:

---
- name: Question
  hosts: localhost
  gather_facts: false
  vars:
    package_key: "ubuntu"
    packages:
        openssh-server:
            archlinux:
                - pkg1
                - pkg2
            ubuntu:
                - pkg3
        cowsay:
            archlinux:
                - pkg4
        app-armor:
            archlinux:
                - pkg5
            ubuntu:
                - pkg6
                - pkg7
  tasks:
    - name: Transformation
      debug:
        msg={{ packages | dict2items | <something> | items2dict }}

If I know the package_key to be ubuntu on such a system, I would like to transform the dictionary so that I get:

    transformed_packages:
        openssh-server:
            - pkg3
        app-armor:
            - pkg6
            - pkg7

What <something> do I need for this transformation? (Where the ubuntu packages are children of the applications, and when there is no ubuntu package, it's not part of the transformed dict).

My reason for this transformation is that I find it more convenient to group per app when I edit what packages are needed for different distros. But at the same time, it's more convenient to just keep what I need when I know the OS.

Answer 1

Create a dictionary of all packages

  package_key: ubuntu
  pkg_attr: "value.{{ package_key }}"
  pkg_all: "{{ dict(packages.keys()|
                    zip(packages|
                        dict2items|
                        map(attribute=pkg_attr, default=[]))) }}"

gives

  pkg_all:
    app-armor: [pkg6, pkg7]
    cowsay: []
    openssh-server: [pkg3]

---

Optionally, you can use json_query

  pkg_query: '[].[key, {{ pkg_attr }}]'
  pkg_all: "{{ dict(packages|dict2items|json_query(pkg_query)) }}"

---

Select non-empty lists

  transformed_packages: "{{ pkg_all|
                            dict2items|
                            selectattr('value')|
                            items2dict }}"

gives what you want

  transformed_packages:
    app-armor: [pkg6, pkg7]
    openssh-server: [pkg3]

---

Example of a complete playbook for testing

- hosts: localhost

  vars:

    package_key: ubuntu
    packages:
      app-armor:
        archlinux: [pkg5]
        ubuntu: [pkg6, pkg7]
      cowsay:
        archlinux: [pkg4]
      openssh-server:
        archlinux: [pkg1, pkg2]
        ubuntu: [pkg3]

    pkg_attr: "value.{{ package_key }}"
    pkg_all: "{{ dict(packages.keys()|
                      zip(packages|
                          dict2items|
                          map(attribute=pkg_attr, default=[]))) }}"
    transformed_packages: "{{ pkg_all|
                              dict2items|
                              selectattr('value')|
                              items2dict }}"

  tasks:

    - debug:
        var: pkg_all|to_yaml
    - debug:
        var: transformed_packages|to_yaml

---

You can make the declaration more robust if you don't trust the method keys() and the filter dict2items provides the same order of keys

  pkg_all: "{{ dict(packages|
                    dict2items|
                    map(attribute='key')|
                    zip(packages|
                        dict2items|
                        map(attribute=pkg_attr, default=[]))) }}"

---

Pros and Cons

You can put options 1,2,3 into the vars and make the code of tasks easier readable

1. json_query

Pros: The most efficient option is json_query. Cons: You have to install JmesPath and learn rather specific syntax

  pkg_attr: "value.{{ package_key }}"
  pkg_query: '[].[key, {{ pkg_attr }}]'
  pkg_all: "{{ dict(packages|dict2items|json_query(pkg_query)) }}"
  tp: "{{ pkg_all|dict2items|selectattr('value')|items2dict }}"

2. Ansible and Jinja filters and functions

Pros: Jinja is available by default. Cons: The pipe of Ansible and Jinja filters, and functions is less efficient

  pkg_attr: "value.{{ package_key }}"
  pkg_all: "{{ dict(packages.keys()|
                    zip(packages|
                        dict2items|
                        map(attribute=pkg_attr, default=[]))) }}"
  tp: "{{ pkg_all|dict2items|selectattr('value')|items2dict }}"

3. Jinja template

Pros: This option is always available. You can use it when Ansible and Jinja functions, filters, and tests can't solve the use case without iteration. Cons: Because of the iteration the efficiency is the worst of all options

  tp: |
      {% filter from_yaml %}
      {% for k,v in packages.items() %}
      {% if package_key in v %}
      {{ k }}: {{ v[package_key] }}
      {% endif %}
      {% endfor %}
      {% endfilter %}

4. Iteration in set_fact task

All pros and cons of 3. apply here with one more Con: You make the tasks more complex. The code is easier to read when the declarations of variables are 'hidden' in vars

  - set_fact:
      tp: "{{ tp|default({})|
              combine({item.key: item.value[package_key]}) }}"
    loop: "{{ packages|
              dict2items |
              selectattr('value.' + package_key, 'defined') }}"