Joining after join with yq

Question

I have the following structure in yml:

metadata:
  name: nws-bot
  version: 0.1.0
  jobs:
  - name: name1
    script: job1.sh
    schedule: 0 20 * * *
  - name: name2
    script: job2.sh
    schedule: 0 22 * * *  

I'm trying to create a structure like this:

name1~job1.sh;name2~job2.sh

The job name and script are separated by a ~, and the jobs are separated by a ;.

I can get this to work using

yq '.metadata.jobs[] | [[.name, .script] | join("~")]' project.yml | yq '. | join(";")'

However I wonder if it can be done with a single yqexpression, instead of piping to yq a second time.

Answer 1

Iterating with .metadata.jobs[] (which is a shortcut for .metadata.jobs | .[]) also destructures the array into its individual items. To re-capture them (meanwhile modified with the first join) back into an array (to be applied to the second join) necessitates the collecting array to be superordinate to the destructuring. Thus, move the iteration inside of it, i.e. pull the opening bracket up front (with the initial traversal to .metadata.jobs included or excluded at your preference):

yq '.metadata.jobs[] | [[.name, .script] | join("~")]' project.yml | yq 'join(";")'
#   ╭──────────────────╯
yq '[.metadata.jobs[] | [.name, .script] | join("~")] | join(";")' project.yml
#   ╰────────────────╮
yq '.metadata.jobs | [.[] | [.name, .script] | join("~")] | join(";")' project.yml

Actually, having [.[] | …] is what map(…) does: applying a filter to the items of an array while retaining the array structure. Using named filters can make code more descriptive and facilitate its readability. Alternatively, there's also the update operator |= which can be used to alter the items of a (generally deeper) structure without destructuring it. In both cases, however, your context here needs to be the array itself, so the initial traversal to .metadata.jobs must be completed prior to their usage (e.g. .metadata.jobs[] |= … would in excess also retain the top level objects, making the second join eventually fail).

yq '.metadata.jobs | map([.name, .script] | join("~")) | join(";")' project.yml
# or
yq '.metadata.jobs | .[] |= ([.name, .script] | join("~")) | join(";")' project.yml

Given your sample input, all of them output:

name1~job1.sh;name2~job2.sh