Search code examples
jsonuniquejq

jq get unique value from two keys


I know to get a unique from one key - unique_by('.[].name)

I want to get output by checking for unique values in two keys

but how to do for two keys like unique_by('.[].name,.[].url') and return the input along with other keys?

#input

[
  {
    "name": "abc",
    "url": "https://aa.com",
    "created_at": "2022-09-30T11:17:33.181Z"
  },
  {
    "name": "bb",
    "url": "https://ddd.com",
    "created_at": "2022-09-30T11:14:33.180Z"
  },
  {
    "name": "abc",
    "url": "https://aa.com",
    "created_at": "2022-09-30T11:14:33.180Z"
  }
]

#expected output

[
  {
    "name": "abc",
    "url": "https://aa.com",
    "created_at": "2022-09-30T11:17:33.181Z"
  },
  {
    "name": "bb",
    "url": "https://ddd.com",
    "created_at": "2022-09-30T11:14:33.180Z"
  }
]

Solution

  • Just provide to unique_by an array with everything included, so that the array must become unique:

    jq 'unique_by([.name, .url])'
    
    [
      {
        "name": "abc",
        "url": "https://aa.com",
        "created_at": "2022-09-30T11:17:33.181Z"
      },
      {
        "name": "bb",
        "url": "https://ddd.com",
        "created_at": "2022-09-30T11:14:33.180Z"
      }
    ]
    

    Demo