how to read a yaml file in bash shell and iterate through a list inside

Question

I have this yaml file I have a yaml file as below:

repos:
- REPO_NAME: A
  INGRESS_SUBDOMAIN: B
  DOCKER_IMAGE_PATH: C
  IMAGE_NAME: D
  MAJOR_VERSION: E
  MINOR_VERSION: F

the name of the file is f.yaml

in bash, I want to read the content of the file and then iterate through each repo, and to verify print the field "a"

I am dong this but it is not working

#!/usr/bin/bash

file_name="f.yaml"
for row in $(yq eval '.repos[]' "$file_name"); do
    REPO_NAME=$(echo "$row" | yq eval '.REPO_NAME' -)
    INGRESS_SUBDOMAIN=$(echo "$row" | yq eval '.INGRESS_SUBDOMAIN' -)
    DOCKER_IMAGE_PATH=$(echo "$row" | yq eval '.DOCKER_IMAGE_PATH' -)
    IMAGE_NAME=$(echo "$row" | yq eval '.IMAGE_NAME' -)
    MAJOR_VERSION=$(echo "$row" | yq eval '.MAJOR_VERSION' -)
    MINOR_VERSION=$(echo "$row" | yq eval '.MINOR_VERSION' -)

    ECHO $INGRESS_SUBDOMAIN
    ECHO $DOCKER_IMAGE_PATH
    ECHO $IMAGE_NAME
    ECHO $MAJOR_VERSION
    ECHO $MINOR_VERSION
done

right now it is printing a bunch of "null". How to fix it?

Answer 1

for row in is unsuitable when you're iterating over values that have spaces, much less newlines, in them. Use yq --nul-output and a BashFAQ #1 while read loop instead:

while IFS= read -r -d '' repo_yaml; do
  repo_name=$(yq eval '.REPO_NAME' <<<"$repo_yaml")
  # ...and so forth
  echo "repo_name is $repo_name" >&2
done < <(yq --nul-output '.repos[]' <"$file_name")

---

Or, more efficiently, but running security risks if you don't trust your data not to be malicious, one can use yq -o=shell to generate an assignment statement that can be evaled in any POSIX-compliant shell to set all your variables at once:

while IFS= read -r -d '' repo_eval; do
  eval "$repo_eval"
  echo "$REPO_NAME"
  echo "$INGRESS_SUBDOMAIN" # etc
done < <(yq -o=shell --nul-output '.repos[]' <"$file_name")

However, this is very ill-advised if your data could contain something like PATH: /uploads/evil.com/rootkit:/bin:/usr/bin; similar risks exist for LD_PRELOAD and other special values. (You could mitigate those risks by having the yq code prefix your key names with a lower-case substring: only all-caps names have special meanings to the shell and other POSIX-compliant OS-provided components).