Search code examples
jsonunixcommand-lineminify

How can I minify JSON in a shell script?


I've been looking for a way to uglify some JSON while in my bash console. This help using it afterward in another command (for example, to pass json inline to httpie)

Giving:

{
    "foo": "lorem",
    "bar": "ipsum"
}

I want to obtain:

{"foo":"lorem","bar":"ipsum"}

NOTE: this question is intentionnaly greatly inspired by it's pretty-print counterpart. However, googling for bash minify json didn't give me a proper result, hence this questions for the minify/uglify.


Solution

  • TL;DR

    no install

    python -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < my.json
    

    very fast (with jj)

    jj -u < my.json
    

    Perf benchmark

    Here's the script, using hyperfine:

    #!/usr/bin/env bash
    
    tmp=$(mktemp json.XXX)
    tmp_md=$(mktemp md.XXX)
    
    trap "rm $tmp $tmp_md" EXIT
    
    cat <<JSON > $tmp
    {
        "foo": "lorem",
        "bar": "ipsum"
    }
    JSON
    hyperfine \
        --export-markdown $tmp_md \
        --warmup 100 \
        "jj -u < $tmp" \
        "yq eval -j -I=0 < $tmp" \
        "xidel -s - -e '\$json' --printed-json-format=compact < $tmp" \
        "jq --compact-output < $tmp" \
        "python3 -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < $tmp" \
        "ruby -r json -e 'j JSON.parse \$stdin.read' < $tmp"
    
    pbcopy < $tmp_md
    
    

    The result on my mac — MacBook Air (M1, 2020), 8 GB:

    Command Mean [ms] Min [ms] Max [ms] Relative
    jj -u < json.p72 1.3 ± 0.2 0.9 2.7 1.00
    yq eval -j -I=0 < json.p72 4.4 ± 0.4 3.8 7.8 3.37 ± 0.65
    xidel -s - -e '$json' --printed-json-format=compact < json.p72 5.5 ± 0.3 5.0 6.5 4.19 ± 0.77
    python3 -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < json.p72 14.0 ± 0.4 13.4 15.0 10.71 ± 1.89
    jq --compact-output < json.p72 14.4 ± 2.0 13.2 33.6 11.02 ± 2.45
    ruby -r json -e 'j JSON.parse $stdin.read' < json.p72 47.3 ± 0.6 46.1 48.5 36.10 ± 6.32

    Result for a large JSON file (14k lines):

    http https://france-geojson.gregoiredavid.fr/repo/regions.geojson | jj -p > $tmp
    
    Command Mean [ms] Min [ms] Max [ms] Relative
    jj -u < json.wFY 3.4 ± 0.7 2.7 12.2 1.00
    jq --compact-output < json.wFY 35.1 ± 0.4 34.5 36.1 10.24 ± 2.23
    python3 -c 'import json, sys;json.dump(json.load(sys.stdin), sys.stdout)' < json.wFY 47.4 ± 0.5 46.3 48.7 13.82 ± 3.01
    xidel -s - -e '$json' --printed-json-format=compact < json.wFY 55.5 ± 1.2 54.7 63.5 16.17 ± 3.53
    ruby -r json -e 'j JSON.parse $stdin.read' < json.wFY 94.9 ± 0.7 93.8 96.8 27.65 ± 6.02
    yq eval -j -I=0 < json.wFY 3087.0 ± 26.6 3049.3 3126.8 899.63 ± 195.81

    And here is the pretty print counterpart benchmark