Search code examples
jsonjmespath

JMESPath array flatten + create key:value pair


I have a JSON object which I'd like to turn into a dictionary.

The object looks like this:

{
  "data": {
    "votes": [
      {
        "id": "0x0d185b44dc6a9e4077d44a29fccc22d1f6238192107d08afa890a1bdddc01e10",
        "voter": "0x4Aad3544bE38067F534b6B516f9E645D495c4062"
      },
      {
        "id": "0xf141aa62dc57b20fb6ea1197df44ba0ab4de166c8c19302553065788812beea1",
        "voter": "0xB01f126C9041CC230e91378d688450Db4c3510cA"
      },
      {
        "id": "0xf141aa62dc57b20fb6ea1197df44ba0ab4de166c8c19302553065788812beea1",
        "voter": "0xB01f126C9041CC230e91378d688450Db4c3510cA"
      },
    ]
  }
}

I'm looking to grab the voter values and create an object like the below.

I'm shooting for the preferred result, but if it's not possible, either OK results will do.

OK results

  "0x4Aad3544bE38067F534b6B516f9E645D495c4062": "1",
  "0xB01f126C9041CC230e91378d688450Db4c3510cA": "1"
}
  "0x4Aad3544bE38067F534b6B516f9E645D495c4062": "1",
  "0xB01f126C9041CC230e91378d688450Db4c3510cA": "1",
  "0xB01f126C9041CC230e91378d688450Db4c3510cA": "1"
}

Preferred result

  "0x4Aad3544bE38067F534b6B516f9E645D495c4062": "1",
  "0xB01f126C9041CC230e91378d688450Db4c3510cA": "2",

Testing so far: Using @|data.votes[].join(',',[voter, '1']) I could get a closeish response e.g. "0x4Aad3544bE38067F534b6B516f9E645D495c4062,1" but not what I'm looking for.

Please can anyone help for either satisfactory result or preferred result?

Edit: Using @|data.votes[].join('":"',[voter, '1']) I now have this which is better.

[
  "0x4Aad3544bE38067F534b6B516f9E645D495c4062\":\"1",
  "0xB01f126C9041CC230e91378d688450Db4c3510cA\":\"1",
  "0xB01f126C9041CC230e91378d688450Db4c3510cA\":\"1"
]

Solution

  • JmesPath is not able to count the frequencies of items in a list. You'll have to use other tool. For example,

    shell> cat data.json | jq '.data.votes[].voter' | uniq -c | awk '{ print $2, $1 }'
    "0x4Aad3544bE38067F534b6B516f9E645D495c4062" 1
    "0xB01f126C9041CC230e91378d688450Db4c3510cA" 2
    

    • Python
    shell> cat count_voters.py 
    #!/usr/bin/python3.9
    
    import jmespath
    import json
    import yaml
    import collections
    
    
    with open('data.json', 'r') as myfile:
        data = myfile.read()
    s = json.loads(data)
    print(yaml.dump(s))
    
    voters = jmespath.search('data.votes[*].voter', s)
    print(yaml.dump(voters))
    
    voters_freq = dict(collections.Counter(voters))
    print(yaml.dump(voters_freq))
    

    gives

    shell> ./count_voters.py
    data:
      votes:
      - id: '0x0d185'
        voter: '0x4Aad3544bE38067F534b6B516f9E645D495c4062'
      - id: '0xf141aa'
        voter: '0xB01f126C9041CC230e91378d688450Db4c3510cA'
      - id: '0xf141aa'
        voter: '0xB01f126C9041CC230e91378d688450Db4c3510cA'
    
    - '0x4Aad3544bE38067F534b6B516f9E645D495c4062'
    - '0xB01f126C9041CC230e91378d688450Db4c3510cA'
    - '0xB01f126C9041CC230e91378d688450Db4c3510cA'
    
    '0x4Aad3544bE38067F534b6B516f9E645D495c4062': 1
    '0xB01f126C9041CC230e91378d688450Db4c3510cA': 2
    

    • Ansible
    shell> cat count_voters.yml
    - hosts: localhost
    
      vars:
    
        voters: "{{ s.data.votes|json_query('[].voter') }}"
        voters_freq: "{{ voters|community.general.counter }}"
    
      tasks:
    
        - include_vars:
            file: data.json
            name: s
        - debug:
            var: s
        - debug:
            var: voters
        - debug:
            var: voters_freq
    

    gives

    shell> ansible-playbook count_voters.yml
    
    PLAY [localhost] *****************************************************************************
    
    TASK [include_vars] **************************************************************************
    ok: [localhost]
    
    TASK [debug] *********************************************************************************
    ok: [localhost] => 
      s:
        data:
          votes:
          - id: '0x0d185'
            voter: '0x4Aad3544bE38067F534b6B516f9E645D495c4062'
          - id: '0xf141aa'
            voter: '0xB01f126C9041CC230e91378d688450Db4c3510cA'
          - id: '0xf141aa'
            voter: '0xB01f126C9041CC230e91378d688450Db4c3510cA'
    
    TASK [debug] *********************************************************************************
    ok: [localhost] => 
      voters:
      - '0x4Aad3544bE38067F534b6B516f9E645D495c4062'
      - '0xB01f126C9041CC230e91378d688450Db4c3510cA'
      - '0xB01f126C9041CC230e91378d688450Db4c3510cA'
    
    TASK [debug] *********************************************************************************
    ok: [localhost] => 
      voters_freq:
        '0x4Aad3544bE38067F534b6B516f9E645D495c4062': 1
        '0xB01f126C9041CC230e91378d688450Db4c3510cA': 2
    
    PLAY RECAP ***********************************************************************************
    localhost: ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0