Search code examples
jsonstreamexport-to-csvjqmissing-data

Print empty string for a missing key and converting the result to CSV


I am trying to convert belowJSON to CSV using jq command but the final CSV is unable to place deviceName field properly as it's missing in some JSON lines.

{
  "id": "ABC",
  "deviceName": "",
  "total": 100,
  "master": 20
}
{
  "id": "ABC",
  "total": 100,
  "master": 20
}

How can i make sure empty value gets when Key is missing ?.

I Tried below command to generate CSV

./jq -r '[.[]] | @csv' > final.csv

But it gives CSV like below as you can see when deviceName key is missing in JSON it's cell shifting left side.

"ABC","",100,20
"ABC",100,20

I want output something like below which adds empty value if deviceName is missing.

"ABC","",100,20
"ABC","",100,20

Solution

  • In jq you can use the alternate operator // that can be used to return default values. E.g. .foo // 1 will evaluate to 1 if there's no .foo element in the input

    Using that and appending an empty string "" if the key is not present, you can do

    jq -r '[.id // "", .deviceName // "", .total // "", .master // ""] | @csv'
    

    Note: The alternate operator .foo // 1 causes the evaluation to 1 for case when the value of foo is null or false. You may wish to modify the above program if your data contains null or false values.