Search code examples
jsonkubernetesbase64jqetcd

ETCD export as json and decode from base64 all key/values to human readable


Is there any easy command line option to export my entire ETCD database to json file but also decode the keys and values automatically from base64?

What I succeeded to the moment is this(example show 1x key/value):

   ./etcdctl get "" --prefix -w json | jq -r ".[] | .[]  "
 {
 "key": "YnktZGV2L21ldGEvc25hcHNob3RzL3Jvb3QtY29vcmQvcGFydGl0aW9ucy80NDAwNDc0MjQ2MTgzNjUxNzAvNDQwMDQ3NDI0NjE4MzY1MTcxX3RzNDQwMDQ5NDg5ODkxODE5NTI0",
"create_revision": 44536,
"mod_revision": 44536,
"version": 1,
"value": "CPOB0OXRmdeNBhIIX2RlZmF1bHQYhIDgxN/V140GIPKB0OXRmdeNBg=="
}

But I need to decode the entire database keys and values to human readable format?

Thanks

P.S. Final solution after @Jeff Mercado help:

1. /etcdctl get "" --prefix -w json | jq '.[]' > etcd_filter.txt
2. Clear output to form array of objects [{},{} ...{}]
3. cat etcd_filter.txt | jq '.[] | (.key, .value) |= @base64d'

jq playground


Solution

  • If the encoded data is a string and not binary data, you can decode it to a UTF-8 string using the @base64d filter. This should be available in jq 1.6.

    $ ./etcdctl ... | jq '.[][] | (.key, .value) |= @base64d'
    {
      "key": "by-dev/meta/snapshots/root-coord/partitions/440047424618365170/440047424618365171_ts440049489891819524",
      "create_revision": 44536,
      "mod_revision": 44536,
      "version": 1,
      "value": "\b���љ׍\u0006\u0012\b_default\u0018������׍\u0006 ���љ׍\u0006"
    }
    

    It appears the value is not a UTF-8 string in your example so beware. Unfortunately, it doesn't return a byte array so it may not be very useful for these cases.

    jqplay