Parse inconsistent JSON file with jq

Question

I want to parse a json file that is the output for the elasticsearch roles I configured on the cluster. Unfortunatelly its structure is different in some places, due to the fact that some role mappings were added manually and some roles mapping were added with API.

example JSON:

{
  "MAPPING_NAME": "FOO_MAPPING",
  "roles": [
    "FOO_ROLE"
  ],
  "rules": {
    "field": {
      "groups": "CN=FO_SEC_GROUP,OU=foo,OU=example,DC=com"
    }
  }
}
{
  "MAPPING_NAME": "BAR_MAPPING",
  "roles": [
    "BAR_ROLE",
    "builtin_role"
  ],
  "rules": {
    "all": [
      {
        "field": {
          "groups": "CN=BAR_SEC_GROUP,OU=bar,OU=example,DC=com"
        }
      }
    ]
  }
}

I want to transform the above json to look like below, but the the 1st section has "rules -> fields -> groups" and the 2nd section has "rules -> all -> fields -> groups"

{
  "MAPPING_NAME": "FOO_MAPPING",
  "roles": [ 
    "FOO_ROLE"
  ],
  "GROUPS": "CN=FO_SEC_GROUP,OU=foo,OU=example,DC=com"
}
{
  "MAPPING_NAME": "BAR_MAPPING",
  "roles": [
    "BAR_ROLE",
    "builtin_role"
  ],
  "GROUPS": "CN=BAR_SEC_GROUP,OU=bar,OU=example,DC=com"
}

Is there any way to parse with jq and eventually to transform to CSV format? Thank you.

Answer 1

You can use has to check for the existence of a key called all:

jq 'del(.rules) + {
  GROUPS: (.rules | if has("all") then .all[] else . end | .field.groups)
}' input.json

{
  "MAPPING_NAME": "FOO_MAPPING",
  "roles": [
    "FOO_ROLE"
  ],
  "GROUPS": "CN=FO_SEC_GROUP,OU=foo,OU=example,DC=com"
}
{
  "MAPPING_NAME": "BAR_MAPPING",
  "roles": [
    "BAR_ROLE",
    "builtin_role"
  ],
  "GROUPS": "CN=BAR_SEC_GROUP,OU=bar,OU=example,DC=com"
}

Demo

Another approach would be to make use of the Error Suppression Operator ? in combination with the Alternative Operator //:

jq 'del(.rules) + {GROUPS: (.rules | .all[]? // . | .field.groups)}'

{
  "MAPPING_NAME": "FOO_MAPPING",
  "roles": [
    "FOO_ROLE"
  ],
  "GROUPS": "CN=FO_SEC_GROUP,OU=foo,OU=example,DC=com"
}
{
  "MAPPING_NAME": "BAR_MAPPING",
  "roles": [
    "BAR_ROLE",
    "builtin_role"
  ],
  "GROUPS": "CN=BAR_SEC_GROUP,OU=bar,OU=example,DC=com"
}

Demo

A similar, yet another approach would be to make use of the Destructuring Alternative Operator ?//...