I would like to combine multiple dict in a list of dicts based on multiple key/value matches.
To be precise: match key value "ru" & "code", if it matches combine the dict and add "usg_amt" of mutiple dict that matches.
Input
{
"cis": [
{
"Id": "388",
"type": "usage",
"properties": {
"usg_amt": "144",
"ru":"123.01",
"code":"2236"
}
},
{
"Id": "389",
"type": "usage",
"properties": {
"usg_amt": "82",
"ru":"123.01",
"code":"2236"
}
},
{
"Id": "19",
"type": "usage",
"properties": {
"usg_amt": "12",
"ru":"124.01",
"code":"2235"
}
}
]
}
Expected
{
"cis": [
{
"Id": "388",
"type": "usage",
"properties": {
"usg_amt": "226", ###### 82+144
"ru":"123.01",
"code":"2236"
}
},
{
"Id": "19",
"type": "usage",
"properties": {
"usg_amt": "12",
"ru":"124.01",
"code":"2235"
}
}
]
}
Code
But the below code is only giving a single dict as output.
- name: Combine the dictionaries that have matching age
set_fact:
combined_list: "{{ datacreate.cis | combine({'key': ['ru', 'code']}) }}"
Q: "Match attributes ru and code. If they match combine the dictionaries."
A: Create the list of the indexes cis_index and combine the dictionaries
cis_index: "{{ cis|json_query(cis_index_query) }}"
cis_index_query: '[].{index: join(`_`, [properties.ru, properties.code])}'
cis_indexs: "{{ cis|zip(cis_index)|map('combine') }}"
give
cis_index:
- index: '123.01_2236'
- index: '123.01_2236'
- index: '124.01_2235'
cis_indexs:
- Id: '388'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: '144'}
type: usage
- Id: '389'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: '82'}
type: usage
- Id: '19'
index: '124.01_2235'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
Group the items of the list by index
cis_groups: "{{ cis_indexs|groupby('index') }}"
gives
cis_groups:
- - '123.01_2236'
- - Id: '388'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: '144'}
type: usage
- Id: '389'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: '82'}
type: usage
- - '124.01_2235'
- - Id: '19'
index: '124.01_2235'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
Create the expected structure in Jinja. Fit the template to your needs
cis_update_str: |
{% for i in cis_groups %}
- {{ i.1|reverse|combine }}
{% endfor %}
cis_update: "{{ cis_update_str|from_yaml }}"
gives
cis_update:
- Id: '388'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: '144'}
type: usage
- Id: '19'
index: '124.01_2235'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
- hosts: localhost
vars:
cis:
- Id: '388'
properties: {code: '2236', ru: '123.01', usg_amt: '144'}
type: usage
- Id: '389'
properties: {code: '2236', ru: '123.01', usg_amt: '82'}
type: usage
- Id: '19'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
cis_index: "{{ cis|json_query(cis_index_query) }}"
cis_index_query: '[].{index: join(`_`, [properties.ru, properties.code])}'
cis_indexs: "{{ cis|zip(cis_index)|map('combine') }}"
cis_groups: "{{ cis_indexs|groupby('index') }}"
cis_update_str: |
{% for i in cis_groups %}
- {{ i.1|reverse|combine }}
{% endfor %}
cis_update: "{{ cis_update_str|from_yaml }}"
cis_updat2: "{{ cis_groups|map('last')|map('reverse')|map('combine') }}"
tasks:
- debug:
var: cis|to_yaml
- debug:
var: cis_index
- debug:
var: cis_indexs|to_yaml
- debug:
var: cis_groups|to_yaml
- debug:
var: cis_update|to_yaml
- debug:
var: cis_updat2|to_yaml
Q: "Sum all attributes usg_amt."
A: Fit the template to your needs
cis_update_str: |
{% for i in cis_groups %}
{% set usg_amt = i.1|map(attribute='properties.usg_amt')|map('int')|sum %}
- {{ i.1|reverse|combine([{'properties': {'usg_amt': usg_amt}}], recursive=true) }}
{% endfor %}
cis_update: "{{ cis_update_str|from_yaml }}"
gives
cis_update:
- Id: '388'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: 226}
type: usage
- Id: '19'
index: '124.01_2235'
properties: {code: '2235', ru: '124.01', usg_amt: 12}
type: usage
Use the filter ansible.utils.remove_keys if you want to remove the attribute index from the result
Q: "This code is not adding the decimal values and it's ignoring all the usg_amt values which are having decimals."
A: Fit the template to your needs
cis_update_str: |
{% for i in cis_groups %}
{% set usg_amt_list = i.1|map(attribute='properties.usg_amt') %}
{% set sum_float = usg_amt_list|select('search', '\.')|length > 0 %}
{% if sum_float %}
{% set usg_amt = usg_amt_list|map('float')|sum %}
{% else %}
{% set usg_amt = usg_amt_list|map('int')|sum %}
{% endif %}
- {{ i.1|reverse|combine([{'properties': {'usg_amt': usg_amt}}], recursive=true) }}
{% endfor %}
cis_update: "{{ cis_update_str|from_yaml }}"
Given the data to test floats (decimals)
cis:
- Id: '388'
properties: {code: '2236', ru: '123.01', usg_amt: '144'}
type: usage
- Id: '389'
properties: {code: '2236', ru: '123.01', usg_amt: '82'}
type: usage
- Id: '19'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
- Id: '20'
properties: {code: '2237', ru: '125.01', usg_amt: '1.10'}
type: usage
- Id: '21'
properties: {code: '2237', ru: '125.01', usg_amt: '1.20'}
type: usage
Update the attribute usg_amt with float if any of the items is float
cis_update:
- Id: '388'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: 226}
type: usage
- Id: '19'
index: '124.01_2235'
properties: {code: '2235', ru: '124.01', usg_amt: 12}
type: usage
- Id: '20'
index: '125.01_2237'
properties: {code: '2237', ru: '125.01', usg_amt: 2.3}
type: usage
In the input list cis the values of the attribute usg_amt are strings. If you want to keep the strings also in the updated list convert the integers, or floats to strings
- {{ i.1|reverse|combine([{'properties': {'usg_amt': usg_amt|string}}], recursive=true) }}
gives
cis_update:
- Id: '388'
index: '123.01_2236'
properties: {code: '2236', ru: '123.01', usg_amt: '226'}
type: usage
- Id: '19'
index: '124.01_2235'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
- Id: '20'
index: '125.01_2237'
properties: {code: '2237', ru: '125.01', usg_amt: '2.3'}
type: usage
Example of a complete playbook for testing
- hosts: localhost
vars:
cis:
- Id: '388'
properties: {code: '2236', ru: '123.01', usg_amt: '144'}
type: usage
- Id: '389'
properties: {code: '2236', ru: '123.01', usg_amt: '82'}
type: usage
- Id: '19'
properties: {code: '2235', ru: '124.01', usg_amt: '12'}
type: usage
- Id: '20'
properties: {code: '2237', ru: '125.01', usg_amt: '1.10'}
type: usage
- Id: '21'
properties: {code: '2237', ru: '125.01', usg_amt: '1.20'}
type: usage
cis_index: "{{ cis|json_query(cis_index_query) }}"
cis_index_query: '[].{index: join(`_`, [properties.ru, properties.code])}'
cis_indexs: "{{ cis|zip(cis_index)|map('combine') }}"
cis_groups: "{{ cis_indexs|groupby('index') }}"
cis_update_str: |
{% for i in cis_groups %}
{% set usg_amt_list = i.1|map(attribute='properties.usg_amt') %}
{% set sum_float = usg_amt_list|select('search', '\.')|length > 0 %}
{% if sum_float %}
{% set usg_amt = usg_amt_list|map('float')|sum %}
{% else %}
{% set usg_amt = usg_amt_list|map('int')|sum %}
{% endif %}
- {{ i.1|reverse|combine([{'properties': {'usg_amt': usg_amt|string}}], recursive=true) }}
{% endfor %}
cis_update: "{{ cis_update_str|from_yaml }}"
tasks:
- debug:
var: cis|to_yaml
- debug:
var: cis_index
- debug:
var: cis_indexs|to_yaml
- debug:
var: cis_groups|to_yaml
- debug:
var: cis_update|to_yaml