Search code examples
jsonvariablesansiblejson-query

ansible json-query trying to select "id" by content file wildcard


Very new to JSON.
I'm trying to extract 2 variables from this json file. It has many files and id's but I only want the file & id if it contains es7.x86_64

When done my desired variables would be:

id=13140

file=NessusAgent-8.3.0-es7.x86_64.rpm

{
    "banners": [],
    "containsRequiredAuth": true,
    "created_at": "2017-10-13T00:53:32.137Z",
    "description": "Download Nessus Agents for use with Tenable.io and Nessus Manager",
    "documentation_link": null,
    "downloads": [
        {
            "created_at": "2021-06-29T19:06:41.776Z",
            "description": "Red Hat ES 7 (64-bit) / CentOS 7 / Oracle Linux 7 (including Unbreakable Enterprise Kernel)",
            "file": "NessusAgent-8.3.0-es7.x86_64.rpm",
            "id": 13140,
            "meta_data": {
                "md5": "f67a2bdd2a7180f66b75f319439d56d5",
                "product": "Nessus Agents - 8.3.0",
                "product_notes": null,
                "product_release_date": "06/29/2021",
                "product_type": "default",
                "release_date": "06/03/2021",
                "sha256": "8a6452086ce0a7193e0f24b1f2adbff3aa6bd0f4ac519384e8453bb68bae0460",
                "version": "8.3.0"
            },
            "name": "NessusAgent-8.3.0-es7.x86_64.rpm",
            "page_id": 61,
            "publish": true,
            "required_auth": false,
            "size": 16375828,
            "sort_order": null,
            "type": "download",
            "updated_at": "2021-06-29T19:08:47.628Z"
        },

My utterly failed attempt to assign file & id variables that have es7.x86_64.

  - name: Convert agent_tempfile to json and register result
    shell: python -m json.tool "{{ agent_tempfile }}"
    register: result

  - name: Extract file & id for es7.x86_64 rpm's
    set_fact:
      agent_id: "{{ result | json_query('downloads[*es7.x86_64*].id') | first }}"
      agent_file: "{{ result | json_query('downloads[*es7.x86_64*].file') | first }}"

I have a feeling I'm going to be doing a lot more of these types of queries soon. Can some one also direct me to a good guide that details parsing specific values from JSON output? The stuff I've found so far just lists arrays but I really want to know how to pull specific data out.


Solution

  • First, there are some great tools out there for playing with JMESPath syntax (the syntax used by the json_query filter). The examples in the JMESPath tutorial are all "live": you can paste your own data into the text fields, and then experiment with filters and check the result.

    The jpterm command is a terminal tool for experimenting with JMESPath queries. This is my personal favorite.


    To look for items that contain a specific substring (like es7.x86_64), you can use the contains operator, like this:

    json_query("downloads[?contains(name, 'es7.x86_64')]")
    

    To make this work for your code, we first need to deal with the fact that the result of your first task is going to be a string, rather than a dictionary. We'll need to pass the standard output through the from_json filter.

    We can also avoid having two almost identical json_query expression by moving the bulk of the expression into a task-local variable.

    This gives us something like:

    - hosts: localhost
      gather_facts: false
      tasks:
        - command: cat data.json
          register: result
    
        - set_fact:
            agent_id: "{{ selected[0].id }}"
            agent_file: "{{ selected[0].file }}"
          vars:
            selected: >-
              {{
                result.stdout |
                from_json |
                json_query("downloads[?contains(name, 'es7.x86_64')]")
              }}
    
        - debug:
            msg:
              - "ID: {{ agent_id }}"
              - "FILE: {{ agent_file }}"
    

    When that task runs, the value of selected will be something like:

    [
      {
        "file": "NessusAgent-8.3.0-es7.x86_64.rpm",
        "id": 13140,
        "name": "NessusAgent-8.3.0-es7.x86_64.rpm",
        "page_id": 61,
        "publish": true,
        "required_auth": false,
        "size": 16375828,
        "sort_order": null,
        "type": "download",
        "updated_at": "2021-06-29T19:08:47.628Z"
      }
    ]
    

    This assumes you're only expecting a single result, so we can just ask for selected[0] to get at that dictionary, and then it's a simple matter of getting at the .id and .file attributes.

    Running the above playbook produces:

    TASK [debug] *********************************************************************************
    ok: [localhost] => {
        "msg": [
            "ID: 13140",
            "FILE: NessusAgent-8.3.0-es7.x86_64.rpm"
        ]
    }