Node.js ReadableStream and Asynchronous Event Listeners

Question

Can someone explain the following behavior? It's certainly due to asynchronous I/O, but the code below is based on many simple examples, some from SO, which apparently don't discuss the issue here, where the stream isn't being read as expected.

And what's the solution? I'm trying to understand the underlying issue from first principles, so please don't advise me to use a published npm stream->string package. Thanks.

Given file n.js

'use strict';

const streamToString = (s, cb) => {
  const chunks = []

  s.on('readable', () => {
    console.log('data')
    let chunk
    while( null !== (chunk = s.read())) {
      chunks.push(chunk)
    }
  })
  s.on('end', () => {
    console.log('end')
    cb(Buffer.concat(chunks).toString())
  })
}

let o
const f = (str) => {
  o = JSON.parse(str)
}

const fs = require('fs')
const rs = fs.createReadStream('t.json')
streamToString(rs, f)

console.log('o = ' + o)

At the terminal

$ uname -a
Linux bsh 4.10.0-40-generic #44~16.04.1-Ubuntu SMP Thu Nov 9 15:37:44 UTC        2017 x86_64 x86_64 x86_64 GNU/Linux
$ node --version
v6.12.0
$ node n.js
o = undefined
data
data
end

The output holds for any non-empty input file, containing simple validated JSON.

I've also tried with the 'read' event, i.e.

  s.on('read', (chunk) => {
    console.log('data')
    chunks.push(chunk)
  })

and the output is

o = undefined
data
end

Answer 1

1: console.log('o = ' + o) is a sync code, so it run before the callback function f you pass to streamToString which is executed async. so when the time console.log('o = ' + o) exceuted, the f function isn't executed yet, this is why o is undefined. Just move the console.log inside the callback function f to get what you want.

const f = (str) => {
  o = JSON.parse(str);
  console.log('o = ' + o);
} 

2: readable is emitted twice during the whole process, and read is emitted only once, see more details here