Search code examples
jsondiffjq

Using jq or alternative command line tools to compare JSON files

Are there any command line utilities that can be used to find if two JSON files are identical with invariance to within-dictionary-key and within-list-element ordering?

Could this be done with jq or some other equivalent tool?

Examples:

These two JSON files are identical

A:

{
  "People": ["John", "Bryan"],
  "City": "Boston",
  "State": "MA"
}

B:

{
  "People": ["Bryan", "John"],
  "State": "MA",
  "City": "Boston"
}

but these two JSON files are different:

A:

{
  "People": ["John", "Bryan", "Carla"],
  "City": "Boston",
  "State": "MA"
}

C:

{
  "People": ["Bryan", "John"],
  "State": "MA",
  "City": "Boston"
}

That would be:

$ some_diff_command A.json B.json

$ some_diff_command A.json C.json
The files are not structurally identical
Solution

  • Since jq's comparison already compares objects without taking into account key ordering, all that's left is to sort all lists inside the object before comparing them. Assuming your two files are named a.json and b.json, on the latest jq nightly:

    jq --argfile a a.json --argfile b b.json -n '($a | (.. | arrays) |= sort) as $a | ($b | (.. | arrays) |= sort) as $b | $a == $b'

    This program should return "true" or "false" depending on whether or not the objects are equal using the definition of equality you ask for.

    EDIT: The (.. | arrays) |= sort construct doesn't actually work as expected on some edge cases. This GitHub issue explains why and provides some alternatives, such as:

    def post_recurse(f): def r: (f | select(. != null) | r), .; r; def post_recurse: post_recurse(.[]?); (post_recurse | arrays) |= sort

    Applied to the jq invocation above:

    jq --argfile a a.json --argfile b b.json -n 'def post_recurse(f): def r: (f | select(. != null) | r), .; r; def post_recurse: post_recurse(.[]?); ($a | (post_recurse | arrays) |= sort) as $a | ($b | (post_recurse | arrays) |= sort) as $b | $a == $b'