Jq fails and shows no output on SSE endpoint through curl

Working example with Jq

echo 'data:{"id":"66fc29f995e5b148fe5c7f7d","summary":"Balans: 305.37","level":"info","date":"2024-10-01T16:57:29.873+00:00"}' | sed 's/data://g' | jq '.summary'

Even with new line at the end

echo 'data:{"id":"66fc29f995e5b148fe5c7f7d","summary":"Balans: 305.37","level":"info","date":"2024-10-01T16:57:29.873+00:00"}
' | sed 's/data://g'  | jq '.summary'

Output (success)

"Balans: 305.37"

Not working example with Curl (and SSE endpoint)

curl -N --http2 -H "Accept:text/event-stream" --silent https://my_domain/stream/OyP1cEC3N7BTQrCsm3AHrTFAj5F5VKCH/notifications | sed 's/data://g' | jq '.summary'

This shows no output! My question is: why is this not working? The following line does however show an output:

Working example with Curl (and SSE endpoint), without jq at end

anon@galliumos:~/Bureaublad$ curl -N --http2 -H "Accept:text/event-stream" --silent https://my_domain/stream/OyP1cEC3N7BTQrCsm3AHrTFAj5F5VKCH/notifications | sed 's/data://g'
{"id":"66fc310d95e5b148fe5c7f8d","summary":"Balans: 305.37","level":"info","date":"2024-10-01T17:27:41.679+00:00"}

The response on my SSE endpoint looks like this, and is working:

data:{"id":"66fc310d95e5b148fe5c7f8d","summary":"Balans: 305.37","level":"info","date":"2024-10-01T17:27:41.679+00:00"}

data:{"id":"66fc310d95e5b148fe5c7f8a","summary":"Balans: 305.37","level":"info","date":"2024-10-01T17:27:41.679+00:00"}

... more records here ...

Update with local SSE test server using NodeJS

This provides a sample SSE endpoint on localhost port 3000.

Create a file named server.js and add the code
Run server with: node server.js

const http = require('http');
const port = process.env.PORT || 3000;

const server = http.createServer((req, res) => {
  // Server-sent events endpoint
  if (req.url === '/events') {
    res.writeHead(200, {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      ...(req.httpVersionMajor === 1 && { 'Connection': 'keep-alive' })
    });

    const refreshRate = 1000; // in milliseconds
    return setInterval(() => {
      const data = '{"id":"66fc310d95e5b148fe5c7f8d","summary":"Balans: 305.37","level":"info","date":"2024-10-01T17:27:41.679+00:00"}';
      const message = `data:${data}\n\n`;
      res.write(message);
    }, refreshRate);
  }

  // Client side
  res.writeHead(200, {'Content-Type': 'text/html'});
  res.end(`
    <!DOCTYPE html>
    <html lang="en" dir="ltr">
      <head>
        <meta charset="utf-8">
        <title>SSE</title>
      </head>
      <body>
        <pre id="log"></pre>
      </body>
      <script>
        var eventSource = new EventSource('/events');
        eventSource.onmessage = function(event) {
          document.getElementById('log').innerHTML += event.data + '<br>';
        };
      </script>
    </html>
  `);
});

server.listen(port);

server.on('error', (err) => {
  console.log(err);
  process.exit(1);
});

server.on('listening', () => {
  console.log(`Listening on port ${port}`);
});

Getting output:

curl -N -H "Accept:text/event-stream" --silent http://localhost:3000/events

Tests results

I got no output/ results with:

curl -N -H "Accept:text/event-stream" --silent http://localhost:3000/events | jq -Rs 'match("data:(.*)")|.captures[0].string|fromjson|.summary'

and

curl -N -H "Accept:text/event-stream" --silent http://localhost:3000/events | tr -cd '[:print:]' | sed 's/^data://' | jq '.summary'

Update 2: Output of `hexdump -C` in combination with curl `--no-buffer`

curl -N --http2 -H "Accept:text/event-stream" --silent --no-buffer  https://my_domain/stream/OyP1cEC3N7BTQrCsm3AHrTFAj5F5VKCH/notifications | hexdump -C

Will give the following output:

00000000  64 61 74 61 3a 7b 22 69  64 22 3a 22 36 36 66 63  |data:{"id":"66fc|
00000010  64 38 39 65 39 35 65 35  62 31 34 38 66 65 35 63  |d89e95e5b148fe5c|
00000020  37 66 64 32 22 2c 22 73  75 6d 6d 61 72 79 22 3a  |7fd2","summary":|
00000030  22 55 6e 69 6e 76 6f 69  63 65 64 20 61 6d 6f 75  |"Uninvoiced amou|
00000040  6e 74 20 30 2e 33 39 22  2c 22 6c 65 76 65 6c 22  |nt 0.39","level"|
00000050  3a 22 69 6e 66 6f 22 2c  22 64 61 74 65 22 3a 22  |:"info","date":"|
00000060  32 30 32 34 2d 31 30 2d  30 32 54 30 35 3a 32 32  |2024-10-02T05:22|
00000070  3a 33 38 2e 38 37 37 2b  30 30 3a 30 30 22 7d 0a  |:38.877+00:00"}.|

Solution

Using your NodeJS test server, this works for me:

$ node server.js &
[1] 691108
Listening on port 3000

$ curl -N -H "Accept:text/event-stream" --silent http://localhost:3000/events \
> | jq -R 'scan("^data:(.*)")[] | fromjson.summary'
"Balans: 305.37"
"Balans: 305.37"
"Balans: 305.37"
"Balans: 305.37"
^C