Search code examples
yamlyq

yq replace all values in a list with decoded version of itself

I have some yaml as below:

- id: 1.3.6.1.4.1.57264.1.1
  critical: false
  value: aHR0cHM6Ly90b2tlbi5hY3Rpb25zLmdpdGh1YnVzZXJjb250ZW50LmNvbQ==
- id: 1.3.6.1.4.1.57264.1.2
  critical: false
  value: c2NoZWR1bGU=
- id: 1.3.6.1.4.1.57264.1.3
  critical: false
  value: NDFkZjA0YjFkZmYzNDQ5MGRiNTg0NDk1ZjZiOTE2YWQxYjBlOGY2ZA==
- id: 1.3.6.1.4.1.57264.1.4
  critical: false
  value: Q3JlYXRlIFJlbGVhc2U=
- id: 1.3.6.1.4.1.57264.1.5
  critical: false
  value: Y2hhaW5ndWFyZC1pbWFnZXMvYWxwaW5lLWJhc2U=
- id: 1.3.6.1.4.1.57264.1.6
  critical: false
  value: cmVmcy9oZWFkcy9tYWlu

How can I use yq to replace the value: <base64_string> with it's decoded value?

The desired output would be:

- id: 1.3.6.1.4.1.57264.1.1
  critical: false
  value: https://token.actions.githubusercontent.com
- id: 1.3.6.1.4.1.57264.1.2
  critical: false
  value: schedule
- id: 1.3.6.1.4.1.57264.1.3
  critical: false
  value: 41df04b1dff34490db584495f6b916ad1b0e8f6d
- id: 1.3.6.1.4.1.57264.1.4
  critical: false
  value: Create Release
- id: 1.3.6.1.4.1.57264.1.5
  critical: false
  value: chainguard-images/alpine-base
- id: 1.3.6.1.4.1.57264.1.6
  critical: false
  value: refs/heads/main

Getting and decoding the base64 is relatively straightforward yq '.[].value | @base64d'

I've tried many combinations of yq syntax but haven't figured this out. The closest I've gotten is doing:

export pathEnv=".[].value"
export valueEnv=".[].value|@base64d"

yq eval 'eval(strenv(pathEnv)) = strenv(valueEnv)'

But that gets me

- id: 1.3.6.1.4.1.57264.1.1
  critical: false
  value: .[].value|@base64d
- id: 1.3.6.1.4.1.57264.1.2
  critical: false
  value: .[].value|@base64d

... (truncated)

It seems like wrapping the value replacement would work.. yq eval 'eval(strenv(pathEnv)) = eval(strenv(valueEnv))'

but that gets Error: unexpected EOF

I've tried using sub() in various ways but no luck..

All the examples I've seen of string replacement / substitution are for when you have just a string to use as replacement. This seems a bit more complex because the replacement has to refer to a previous value in yq's pipeline somehow, that's then run through @base64d.

Perhaps it's possible using multiple yaml files?

What would be even better is if there was another yaml file mapping the id to it's name, e.g.

oidcIssuer: 1.3.6.1.4.1.57264.1.1
githubWorkflowTrigger: 1.3.6.1.4.1.57264.1.2

and have yq add them, something like:

- id: 1.3.6.1.4.1.57264.1.1
  named_oid: oidcIssuer
  critical: false
  value: https://token.actions.githubusercontent.com
- id: 1.3.6.1.4.1.57264.1.2
  named_oid: githubWorkflowTrigger
  critical: false
  value: schedule
...
Solution

  • Its pretty simple, use the |= operator, which updates back the value to then node being operated, i.e.

    yq '.[].value |= @base64d' yaml

# or alternatively

yq 'map(.value |= @base64d)' yaml

    Note: If you are using yq version 4.18.1 or beyond, the eval flag e is no longer needed as it has been made the default action.