Search code examples
ansibleflattenjmespath

JMESPath or Ansible flatten dictionaries


I would like to transform an input like this:

{
    "cluster1": {
        "services": {
            "service1": {
                "name": "foo",
                "version": 1.0
            },
            "service2": {
                "name": "bar",
                "version": 2.0
            }
        }
    },
    "cluster2": {
        "services": {
            "service3": {
                "name": "test",
                "version": 3.0
            },
            "service4": {
                "name": "s4",
                "version": 4.0
            }
        }
    },
    "cluster3": {
        "services": {
            "service5": {
                "name": "s5",
                "version": 5.0
            }
        }
    },
    "clusterx": {
        "name": "value"
    }
}

or this (doesn't matter which one):

[
    {
        "key": "cluster1",
        "value": {
            "services": {
                "service1": {
                    "name": "foo",
                    "version": 1.0
                },
                "service2": {
                    "name": "bar",
                    "version": 2.0
                }
            }
        }
    },
    {
        "key": "cluster2",
        "value": {
            "services": {
                "service3": {
                    "name": "test",
                    "version": 3.0
                },
                "service4": {
                    "name": "s4",
                    "version": 4.0
                }
            }
        }
    },
    {
        "key": "cluster3",
        "value": {
            "services": {
                "service5": {
                    "name": "s5",
                    "version": 5.0
                }
            }
        }
    },
    {
        "key": "clusterx",
        "value": {
            "name": "value"
        }
    }
]

into this:

[
  { "name": "service1", "value": {
    "name": "foo",
    "version": 1
  },
  { "name": "service2", "value": {
    "name": "bar",
    "version": 2
  },
  { "name": "service3", "value": {
    "name": "test",
    "version": 3
  },
  { "name": "service4", "value": {
    "name": "s4",
    "version": 4
  },
  { "name": "service5", "value": {
    "name": "s5",
    "version": 5
  }
]

So basically I would like to see a list of all the services flattened.

Is it possible to do this with JMESPath?

What I have so far for the 2nd input type is this: [?value.services].value.services which transforms my input into this:

[
  {
    "service1": {
      "name": "foo",
      "version": 1
    },
    "service2": {
      "name": "bar",
      "version": 2
    }
  },
  {
    "service3": {
      "name": "test",
      "version": 3
    },
    "service4": {
      "name": "s4",
      "version": 4
    }
  },
  {
    "service5": {
      "name": "s5",
      "version": 5
    }
  }
]

So I guess I need another level of flattening, but I'm not sure how to do it.

I'm using Ansible by the way where json_query is very similar to JMESPath. So if we can achieve this with Ansible / json_query, that's also perfect.

In Ansible I tried something like this which is almost there, but not quite:

all_services: |
  {{ input | dict2items | json_query('[?value.services].value.services') }}

Solution

  • Let's combine a flat dictionary of the services first. For example

        - set_fact:
            all_services_dict: "{{ all_services_dict|default({})|combine(item) }}"
          loop: "{{ input|json_query('*.services') }}"
        - debug:
            var: all_services_dict
    

    gives

    
        "all_services_dict": {
            "service1": {
                "name": "foo",
                "version": 1.0
            },
            "service2": {
                "name": "bar",
                "version": 2.0
            },
            "service3": {
                "name": "test",
                "version": 3.0
            },
            "service4": {
                "name": "s4",
                "version": 4.0
            },
            "service5": {
                "name": "s5",
                "version": 5.0
            }
        }
    

    Then concatenate the services into a list. For example

        - set_fact:
            all_services_list: "{{ all_services_list|default([]) +
                                   [{'name': item.key,
                                     'value': item.value}] }}"
          loop: "{{ all_services_dict|dict2items }}"
        - debug:
            var: all_services_list
    

    gives

    
        "all_services_list": [
            {
                "name": "service1",
                "value": {
                    "name": "foo",
                    "version": 1.0
                }
            },
            {
                "name": "service2",
                "value": {
                    "name": "bar",
                    "version": 2.0
                }
            },
            {
                "name": "service3",
                "value": {
                    "name": "test",
                    "version": 3.0
                }
            },
            {
                "name": "service4",
                "value": {
                    "name": "s4",
                    "version": 4.0
                }
            },
            {
                "name": "service5",
                "value": {
                    "name": "s5",
                    "version": 5.0
                }
            }
        ]