Search code examples
jsonstructurejqtransform

How to transform a json structure with jq


How can I transform (using jq) the following json structure from this:

{
  "_internal_messages": {
    "error": [
      {
        "date": "16:12:30 - 07/02/2023",
        "id": 1,
        "origin": "A",
        "text": "This is an error message"
      },
      {
        "date": "16:12:31 - 07/02/2023",
        "id": 5,
        "origin": "A",
        "text": "This is a second error message"
      }
    ],
    "info": [
      {
        "date": "16:12:29 - 07/02/2023",
        "id": 0,
        "origin": "A",
        "text": "This is an info message"
      },
      {
        "date": "16:12:30 - 07/02/2023",
        "id": 4,
        "origin": "C",
        "text": "This is a second info message"
      }
    ],
    "success": [
      {
        "date": "16:12:30 - 07/02/2023",
        "id": 2,
        "origin": "B",
        "text": "This is a success message"
      },
      {
        "date": "16:12:30 - 07/02/2023",
        "id": 3,
        "origin": "B",
        "text": "This is a second success message"
      },
      {
        "date": "16:12:31 - 07/02/2023",
        "id": 6,
        "origin": "C",
        "text": "This is a third success message"
      }
    ]
  }
}

To this:

{"_internal_messages":[
  {
    "type":"info",
    "date": "16:12:29 - 07/02/2023",
    "id": 0,
    "origin": "A",
    "text": "This is an info message"
  },{
    "type":"error",
    "date": "16:12:30 - 07/02/2023",
    "id": 1,
    "origin": "A",
    "text": "This is an error message"
  },{
    "type":"success",
    "date": "16:12:30 - 07/02/2023",
    "id": 2,
    "origin": "B",
    "text": "This is a success message"
  },{
    "type":"success",
    "date": "16:12:30 - 07/02/2023",
    "id": 3,
    "origin": "B",
    "text": "This is a second success message"
  },{
    "type":"info",
    "date": "16:12:30 - 07/02/2023",
    "id": 4,
    "origin": "C",
    "text": "This is a second info message"
  },{
    "type":"error",
    "date": "16:12:31 - 07/02/2023",
    "id": 5,
    "origin": "A",
    "text": "This is a second error message"
  },{
    "type":"success",
    "date": "16:12:31 - 07/02/2023",
    "id": 6,
    "origin": "C",
    "text": "This is a third success message"
  }
]}

I checked the jq Manual and some other previous answered questions at SO, but I was not able to crack this one out... I´m thinking in combining jq with bash to do it, but I´m sure there must be a better way using just jq but my skills are not there yet. Can someone help me out, please? Thanks!

In the original structure the messages are stored in 3 arrays: error, info and success. Each time a message is added to those arrays, it receives a consecutive/incremental id. What I want is to take all those messages out from the three arrays, put them in one array while adding an extra attribute to know where each message came from (error, info or success) and finally order them in asc order by their id. The tricky part is knowing which one is an error, info or a success once you put them all together.


Solution

  • I don't understand why one of the items is missing in your desired output, but apart from that it's flattening the object with mapping the key to another field:

    jq '._internal_messages |= (to_entries | map({type: .key} + .value[]))'
    
    {
      "_internal_messages": [
        {
          "type": "error",
          "date": "16:12:30 - 07/02/2023",
          "id": 1,
          "origin": "A",
          "text": "This is an error message"
        },
        {
          "type": "error",
          "date": "16:12:31 - 07/02/2023",
          "id": 5,
          "origin": "A",
          "text": "This is a second error message"
        },
        {
          "type": "info",
          "date": "16:12:29 - 07/02/2023",
          "id": 0,
          "origin": "A",
          "text": "This is an info message"
        },
        {
          "type": "info",
          "date": "16:12:30 - 07/02/2023",
          "id": 4,
          "origin": "C",
          "text": "This is a second info message"
        },
        {
          "type": "success",
          "date": "16:12:30 - 07/02/2023",
          "id": 2,
          "origin": "B",
          "text": "This is a success message"
        },
        {
          "type": "success",
          "date": "16:12:30 - 07/02/2023",
          "id": 3,
          "origin": "B",
          "text": "This is a second success message"
        },
        {
          "type": "success",
          "date": "16:12:31 - 07/02/2023",
          "id": 6,
          "origin": "C",
          "text": "This is a third success message"
        }
      ]
    }
    

    If you want the items to be sorted by .id, append a sort_by:

    jq '._internal_messages |= (to_entries | map({type: .key} + .value[]) | sort_by(.id))'
    
    {
      "_internal_messages": [
        {
          "type": "info",
          "date": "16:12:29 - 07/02/2023",
          "id": 0,
          "origin": "A",
          "text": "This is an info message"
        },
        {
          "type": "error",
          "date": "16:12:30 - 07/02/2023",
          "id": 1,
          "origin": "A",
          "text": "This is an error message"
        },
        {
          "type": "success",
          "date": "16:12:30 - 07/02/2023",
          "id": 2,
          "origin": "B",
          "text": "This is a success message"
        },
        {
          "type": "success",
          "date": "16:12:30 - 07/02/2023",
          "id": 3,
          "origin": "B",
          "text": "This is a second success message"
        },
        {
          "type": "info",
          "date": "16:12:30 - 07/02/2023",
          "id": 4,
          "origin": "C",
          "text": "This is a second info message"
        },
        {
          "type": "error",
          "date": "16:12:31 - 07/02/2023",
          "id": 5,
          "origin": "A",
          "text": "This is a second error message"
        },
        {
          "type": "success",
          "date": "16:12:31 - 07/02/2023",
          "id": 6,
          "origin": "C",
          "text": "This is a third success message"
        }
      ]
    }