How to convert JSON subelement strings to real JSON elements in bash

I have JSON message exported from Cassandra which I would like to load into Elastic as structured document:

{
  "correlationId": "fb8f855a0eac8985d430896d",
  "leg": 65535,
  "tag": "circuitpath",
  "offset": 479306,
  "len": 508,
  "prev": {
    "page": {
      "file": 10342,
      "page": 8
    },
    "record": 216
  },
  "data": "[ { \"policy\": \"Health Check\", \"execTime\": 0, \"filters\": [  { \"espk\": \"DEFAULT_PRIMARY_VordelGateway_7.4.0:223\", \"name\": \"Set Message\", \"type\": \"ChangeMessageFilter\", \"class\": \"com.vordel.circuit.conversion.ChangeMessageFilter\", \"status\": \"Pass\", \"filterTime\": 1518702587006, \"execTime\": 0 } , { \"espk\": \"DEFAULT_PRIMARY_VordelGateway_7.4.0:222\", \"name\": \"Reflect\", \"type\": \"ReflectFilter\", \"class\": \"com.vordel.circuit.net.ReflectFilter\", \"status\": \"Pass\", \"filterTime\": 1518702587006, \"execTime\": 0 }  ] } ]"
}

But I do not know how by simply way I could convert the "data" content to format which will allow to load it into Elastic.

I have tried following method where $json is the message above:

json2="${json//\\\"/\"}"
json2="${json2//\\\\/\\}"
echo "$json2"

the "data" element then seems like:

"data":"[ { "policy": "Health Check", "execTime": 0, "filters": [ { "espk": "DEFAULT_PRIMARY_VordelGateway_7.4.0:223", "name": "Set Message", "type": "ChangeMessageFilter", "class": "com.vordel.circuit.conversion.ChangeMessageFilter", "status": "Pass", "filterTime": 1518709297006, "execTime": 0 } , { "espk": "DEFAULT_PRIMARY_VordelGateway_7.4.0:222", "name": "Reflect", "type": "ReflectFilter", "class": "com.vordel.circuit.net.ReflectFilter", "status": "Pass", "filterTime": 1518709297006, "execTime": 0 } ] } ]"

but loader is telling me

"status":400,"error":{"type":"mapper_parsing_exception","reason":"failed to parse","caused_by":{"type":"json_parse_exception","reason":"Unexpected character ('p' (code 112)): was expecting comma to separate OBJECT entries\n at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@3508edee; line: 1, column: 170

It seems like it has problem with letter "p" in the first sub-sub-element name "policy"

Any idea how to get it into Elastic?

Solution

To convert the .data element from a JSON string into a JSON object, you could use the filter:

.data |= fromjson

If you just want to extract the .data element and convert it, you could use the filter:

.data | fromjson

For example: jq -c '.data|fromjson' data.json

Supplemental question

If it is uncertain whether or not fromjson will succeed, you could use the idiom: fromjson? // ., e.g.:

.data |= (fromjson? // .)