Search code examples
dictionaryansiblecomparison

Ansible - Combine key:values from lists of dicts


I've two data structures, each a list of dictionaries.

"ldap_identity_sources": [
    {
        "_create_time": 1676110324694,
        "_create_user": "admin",
        "_last_modified_time": 1676110742884,
        "_last_modified_user": "admin",
        "_protection": "NOT_PROTECTED",
        "_revision": 1,
        "_system_owned": false,
        "base_dn": "dc=apac,dc=acme,dc=com",
        "display_name": "apac.acme.com",
        "domain_name": "apac.acme.com",
        "id": "831a1a64-xyz1-2b12-813d-12358ce0b237",
        "ldap_servers": [
            {
                "bind_identity": "[email protected]",
                "enabled": true,
                "url": "LDAP://DCS16Apac02.apac.acme.com:3268",
                "use_starttls": false
            }
        ],
        "resource_type": "ActiveDirectoryIdentitySource"
    },
    {
        "_create_time": 1679110539836,
        "_create_user": "admin",
        "_last_modified_time": 1667677961879,
        "_last_modified_user": "admin",
        "_protection": "NOT_PROTECTED",
        "_revision": 1,
        "_system_owned": false,
        "base_dn": "dc=emea,dc=acme,dc=com",
        "display_name": "emea.acme.com",
        "domain_name": "emea.adme.com",
        "id": "9af347b8-123a-bcd4-958e-abcdefg0060",
        "ldap_servers": [
            {
                "bind_identity": "[email protected]",
                "enabled": true,
                "url": "LDAP://DCS16emea01.emea.acme.com:3268",
                "use_starttls": false
            }
        ],
        "resource_type": "ActiveDirectoryIdentitySource"
    }
    ]

and

"server_cert": [
    {
        "cert": "-----BEGIN CERTIFICATE-----\nBOGUS_CERT_DATA\n-----END CERTIFICATE-----\n",
        "ldap_server": "DCS16APAC02.apac.acme.com"
    }
    ]

I need to add the certificate from "server_cert" into "ldap_identity_sources" by checking to see if the value of server_cert.ldap_server is in the value of "ldap_identity_sources.0.ldap_servers.0.url

That is, is DCS16APAC02.apac.acme.com in LDAP://DCS16Apac02.apac.acme.com:3268

I've tried the following

- name: add Certificate to ldap_identity_sources by comparing values
  set_fact:
    ldap_identity_sources_new: "{{ ldap_identity_sources + [{ 'cert': item.cert }] }}"
  with_items: "{{server_cert}}"
  when: item.ldap_server in ldap_identity_sources.0.ldap_servers.0.url

The above fails with "skip_reason": "Conditional result was False"


Thanks for the below answers but, they are not quite what i'm looking for. That said, they have been helpful in getting closer to what i want.

- name: Add LDAP Servername to ldap_identity_sources
  vars:
    ldap_servername: "{{ item | combine({ 'ldap_servername': item.ldap_servers.0.url[7:-5]|lower }) }}"
  loop: "{{ldap_identity_sources}}"
  set_fact:
    ldap_identity_sources_with_servername: "{{ ldap_identity_sources_with_servername | d([]) + [ldap_servername] }}"

gives

"ldap_identity_sources_with_servername": [
    {
        "_create_time": 1676110324694,
        "_create_user": "admin",
        "_last_modified_time": 1676110742884,
        "_last_modified_user": "admin",
        "_protection": "NOT_PROTECTED",
        "_revision": 1,
        "_system_owned": false,
        "base_dn": "dc=apac,dc=acme,dc=com",
        "display_name": "apac.acme.com",
        "domain_name": "apac.acme.com",
        "id": "831a1a64-xyz1-2b12-813d-12358ce0b237",
        "ldap_servername": "dcs16apac02.apac.acme.com",
        "ldap_servers": [
            {
                "bind_identity": "[email protected]",
                "enabled": true,
                "url": "LDAP://DCS16Apac02.apac.acme.com:3268",
                "use_starttls": false
            }
        ],
        "resource_type": "ActiveDirectoryIdentitySource"
    },
    {
        "_create_time": 1679110539836,
        "_create_user": "admin",
        "_last_modified_time": 1667677961879,
        "_last_modified_user": "admin",
        "_protection": "NOT_PROTECTED",
        "_revision": 1,
        "_system_owned": false,
        "base_dn": "dc=emea,dc=acme,dc=com",
        "display_name": "emea.acme.com",
        "domain_name": "emea.adme.com",
        "id": "9af347b8-123a-bcd4-958e-abcdefg0060",
        "ldap_servername": "dcs16emea01.emea.acme.com",
        "ldap_servers": [
            {
                "bind_identity": "[email protected]",
                "enabled": true,
                "url": "LDAP://DCS16emea01.emea.acme.com:3268",
                "use_starttls": false
            }
        ],
        "resource_type": "ActiveDirectoryIdentitySource"
    }
]

I then convert the server_cert_list to a dict and then to lowercase

- name: convert server_cert_list to a dictionary
  set_fact:
    server_cert_dict: "{{ server_cert_list | items2dict( key_name='ldap_server', value_name='cert' ) }}"

- name:  Convert the server_cert_dict keys to lower-case
  set_fact:
    server_cert_lower: "{{ dict( server_cert_dict.keys()|map('lower') | zip( server_cert_dict.values() ) ) }}"

That gives

"server_cert_lower": {
    "dcs16apac02.apac.acme.com": "-----BEGIN CERTIFICATE-----BOGUS_CERT_DATA-----END CERTIFICATE-----\n",
    "dcs16emea01.emea.acme.com": "-----BEGIN CERTIFICATE-----BOGUS_CERT_DATA-----END CERTIFICATE-----\n"
}

I still need to add the certificate for each server into the list of dicts ldap_identity_sources_with_servername

Now that ldap_identity_sources_with_servername has a key:value for ldap_servername and, server_cert_lower has a key for the servername, i think i'm a step closer to combining the two.

- name: Use Jinja2 to create needed data structure
  set_fact: 
    result_str: |
      {% for e in ldap_identity_sources_with_servername %}
      {{ e }}:
      {% for k,v in e.items() %}
      {% if k in server_cert_lower.keys() %}
        - "ldap_cert": {{server_cert_lower[k]}}
      {% endif %}
      {% endfor %}
      {% endfor %}

Solution

  • Figured it out, combining help from comments provided here and a suggestion from a colleague.

    First, as before, add the servername to the list of Identity Sources. This time, make sure the servername is lowercase

    - name: Add LDAP Servername to ldap_identity_sources
      vars:
        ldap_servername: "{{ item | combine({ 'ldap_server': item.ldap_servers.0.url[7:-5]|lower }) }}"
      loop: "{{ldap_identity_sources}}"
      set_fact:
        ldap_identity_sources_with_servername: "{{ ldap_identity_sources_with_servername | d([]) + [ldap_servername] }}"
    

    That gives

    "ldap_identity_sources": [
        {
            "_create_time": 1676110324694,
            "_create_user": "admin",
            "_last_modified_time": 1676110742884,
            "_last_modified_user": "admin",
            "_protection": "NOT_PROTECTED",
            "_revision": 1,
            "_system_owned": false,
            "base_dn": "dc=apac,dc=acme,dc=com",
            "display_name": "apac.acme.com",
            "domain_name": "apac.acme.com",
            "id": "831a1a64-xyz1-2b12-813d-12358ce0b237",
            "ldap_server": "dcs16apac02.apac.acme.com",
            "ldap_servers": [
                {
                    "bind_identity": "[email protected]",
                    "enabled": true,
                    "url": "LDAP://DCS16Apac02.apac.acme.com:3268",
                    "use_starttls": false
                }
            ],
            "resource_type": "ActiveDirectoryIdentitySource"
        },
        {
            "_create_time": 1679110539836,
            "_create_user": "admin",
            "_last_modified_time": 1667677961879,
            "_last_modified_user": "admin",
            "_protection": "NOT_PROTECTED",
            "_revision": 1,
            "_system_owned": false,
            "base_dn": "dc=emea,dc=acme,dc=com",
            "display_name": "emea.acme.com",
            "domain_name": "emea.acme.com",
            "id": "9af347b8-123a-bcd4-958e-abcdefg0060",
            "ldap_server": "dcs16emea01.emea.acme.com",
            "ldap_servers": [
                {
                    "bind_identity": "[email protected]",
                    "enabled": true,
                    "url": "LDAP://DCS16emea01.emea.acme.com:3268",
                    "use_starttls": false
                }
            ],
            "resource_type": "ActiveDirectoryIdentitySource"
        }
    ]
    

    As before, pair the servername with it's certificate

    - name: Set fact to pair LDAP Server with its Certificate
      set_fact:
        server_cert_list: "{{ server_cert_list|d([]) + [{'ldap_server': item.json.details.0.subject_cn|lower, 'cert': item.json.pem_encoded }] }}"
      with_items: "{{ cert_facts.results }}"
    

    This gives

    "server_cert_list": {
        "dcs16apac02.apac.acme.com": "-----BEGIN CERTIFICATE-----BOGUS_CERT_DATA-----END CERTIFICATE-----\n",
        "dcs16emea01.emea.acme.com": "-----BEGIN CERTIFICATE-----BOGUS_CERT_DATA-----END CERTIFICATE-----\n"
    }
    
    

    Now the solution

    - name: Add LDAP Certificate to ldap_identity_sources_with_servername
      set_fact:
        merged: "{{ merged|d([]) + [dict(item.0, **item.1)] }}"
      with_together:
        - "{{ ldap_identity_sources_with_servername }}"
        - "{{ server_cert_list }}"
      when: item.0.ldap_server == item.1.ldap_server
    
    - name: print merged 
      debug:
        msg: "{{merged}}"
    

    that gives

    "merged": [
        {
            "_create_time": 1676110324694,
            "_create_user": "admin",
            "_last_modified_time": 1676110742884,
            "_last_modified_user": "admin",
            "_protection": "NOT_PROTECTED",
            "_revision": 1,
            "_system_owned": false,
            "base_dn": "dc=apac,dc=acme,dc=com",
            "display_name": "apac.acme.com",
            "domain_name": "apac.acme.com",
            "id": "831a1a64-xyz1-2b12-813d-12358ce0b237",
            "ldap_server": "dcs16apac02.apac.acme.com",
            "cert": "-----BEGIN CERTIFICATE-----BOGUS_CERT_DATA-----END CERTIFICATE-----\n",
            "ldap_servers": [
                {
                    "bind_identity": "[email protected]",
                    "enabled": true,
                    "url": "LDAP://DCS16Apac02.apac.acme.com:3268",
                    "use_starttls": false
                }
            ],
            "resource_type": "ActiveDirectoryIdentitySource"
        },
        {
            "_create_time": 1679110539836,
            "_create_user": "admin",
            "_last_modified_time": 1667677961879,
            "_last_modified_user": "admin",
            "_protection": "NOT_PROTECTED",
            "_revision": 1,
            "_system_owned": false,
            "base_dn": "dc=emea,dc=acme,dc=com",
            "display_name": "emea.acme.com",
            "domain_name": "emea.acme.com",
            "id": "9af347b8-123a-bcd4-958e-abcdefg0060",
            "ldap_server": "dcs16emea01.emea.acme.com",
            "cert": "-----BEGIN CERTIFICATE-----BOGUS_CERT_DATA-----END CERTIFICATE-----\n",
            "ldap_servers": [
                {
                    "bind_identity": "[email protected]",
                    "enabled": true,
                    "url": "LDAP://DCS16emea01.emea.acme.com:3268",
                    "use_starttls": false
                }
            ],
            "resource_type": "ActiveDirectoryIdentitySource"
        }
    ]