Search code examples
linuxbashshellyamlyq

How to format output with spaces inside a shell script?


I've a folder more than 100 json files, I'm trying to create a shell script which will merge a yaml file and a json file together and create yaml file for each json Please find the script below.

json file

{
  "meta": {
    "type": "db",
    "canSave": false,
    "canEdit": false,
    "canAdmin": false,
    "canStar": true,
    "slug": "dashboard-name"
  },
  "dashboard": {
    "annotations": {
      "list": [
        {
          "builtIn": 1,
          "datasource": "-- Grafana --",
          "enable": true,
          "hide": true,
          "iconColor": "rgba(0, 211, 255, 1)",
          "name": "Annotations & Alerts",
          "type": "dashboard"
        }
      ]
    }
  }
}

yaml file

apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata:
  name: testname
  labels:
    app: grafana
    type: dashboard
    folder: proj

Shell Script

#!/bin/bash
for file in *.json
do 
  slug=$(jq -r ".meta.slug" "$file")
  json=$(jq -r ".dashboard" "$file")
  cat << EOF > "$slug.yaml" | yq
  apiVersion: integreatly.org/v1alpha1
  kind: GrafanaDashboard
  metadata:
    name: $slug
    labels:
      app: grafana
      type: dashboard
  spec:
    json: >
      {
        $json
      }
       
EOF
done

yq command I used in shell script earlier.

json=$(yq -o=json -I=4 ".dashboard[]" $file).

This is the command I'm trying to use but it is throwing the below error.

jq: Unknown option -o=json
Use jq --help for help with command-line options,
or see the jq manpage, or online docs  at https://stedolan.github.io/jq
jq: Unknown option -o=json
Use jq --help for help with command-line options,
or see the jq manpage, or online docs  at https://stedolan.github.io/jq
yq: Error running jq: BrokenPipeError: [Errno 32] Broken pipe.
jq: Unknown option -o=json
Use jq --help for help with command-line options,
or see the jq manpage, or online docs  at https://stedolan.github.io/jq
I want the output as shown below.

Requried output:

apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata:
  name: test
  labels:
    app: grafana
    type: dashboard
    folder: proj
spec:
  json: >
    "meta": {
    "type": "db",
    "slug": "test",
    "expires": "0001-01-01T00:00:00Z",
    "created": "2016-05-11T22:00:29Z",
    "updated": "2021-11-29T23:36:37Z",
    "createdBy": "Anonymous"
  }

I am getting the output and the files as shown below

  apiVersion: integreatly.org/v1alpha1
  kind: GrafanaDashboard
  metadata:
    name: dashboard-name
    labels:
      app: grafana
      type: dashboard
      folder: shared-services
  spec:
    json: >
      {
        {
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  }
}
      }
       

I'm not able to get the spacings correctly for all lines as per yaml formatting. Can someone please help me regarding this.


Solution

  • I was able o achieve the required result using.

    json=$(yq -o j '.dashboard' $file | ( SPACE=$'\     ' ; sed "s/^/$SPACE/" ))