Merging Instana EUM JSONs with JQ in CMD

Question

I'm using Instana to deliver view stats on my site, each daily file looks like this:

{
  "items" : [ {
    "name" : "page1.htm",
    "earliestTimestamp" : 1675222177839,
    "cursor" : {
      "@class" : ".IngestionOffsetCursor",
      "ingestionTime" : 1675292168217,
      "offset" : 1
    },
    "metrics" : {
      "uniqueSessions.distinct_count" : [ [ 1675292400000, 4.0 ] ]
    }
  }, {
    "name" : "page2.htm",
    "earliestTimestamp" : 1675260035165,
    "cursor" : {
      "@class" : ".IngestionOffsetCursor",
      "ingestionTime" : 1675292168217,
      "offset" : 2
    },
    "metrics" : {
      "uniqueSessions.distinct_count" : [ [ 1675292400000, 1.0 ] ]
    }
  }, {
    "name" : "page3.htm",
    "earliestTimestamp" : 1675228447118,
    "cursor" : {
      "@class" : ".IngestionOffsetCursor",
      "ingestionTime" : 1675292168217,
      "offset" : 3
    },
    "metrics" : {
      "uniqueSessions.distinct_count" : [ [ 1675292400000, 7.0 ] ]
    }
  } ],
  "canLoadMore" : false,
  "totalHits" : 12,
  "totalRepresentedItemCount" : 12,
  "totalRetainedItemCount" : 12,
  "adjustedTimeframe" : {
    "windowSize" : 86400000,
    "to" : 1675292400000
  }
}

These daily files should be merged into one json after filtering for the necessary info:

• url (from name)

• date (first value in "uniqueSessions.distinct_count")

• number of page visits: (second value in "uniqueSessions.distinct_count")
  It is important, that it has to be done in CMD, since I have to use a batch file as the target user is not allowed to run PowerShell scripts, nor have access to any other CL tool.

So far, I managed to boil down the files to the needed data elements as separate JSON objects using: type *.json | jq ".items[] | {url: .name, date: .metrics[][0][0], load: .metrics[][0][1]}"

the result looks like:

{
  "url": "page1.htm",
  "date": 1675292400000,
  "load": 4
}
{
  "url": "page1.htm",
  "date": 1675292400000,
  "load": 1
}
{
  "url": "page1.htm",
  "date": 1675292400000,
  "load": 7
}

however, if I try to wrap it in square brackets (as tutorials suggest) to get a valid JSON, I get one file with a bunch of arrays starting and ending where they did in the original files.
I did the homework, and am aware of this: combining multiple json files into a single json file with jq filters actually, I played around with this for a while now before asking. I was thinking if I could add again curly brackets and a root node, it would help, but I haven't found a way where JQ wouldn't fail to do noting, that most probably the error comes from windows cmd's quotation mark usage.

How can I make this into one JSON instead of as many arrays as many source files? Thanks!

Answer 1

For multiple input files, you can create another array around all of them using the --slurp (or -s option), then use map on that:

jq -s 'map(.items[] | {…})' *.json

Demo

Or programmatically iterate (e.g.using reduce) over each input (using inputs in combination with the --null-input (or -n) flag):

jq -n 'reduce inputs as {$items} ([]; . + [$items[] | {…}])' *.json

Demo