How to use Streams2 ObjectMode?

Question

In Node v10.11, I'm trying to push objects down a pipe, but i always get error.

events.js:72
    throw er; // Unhandled 'error' event
          ^
TypeError: Invalid non-string/buffer chunk
    at validChunk (_stream_writable.js:150:14)
    at WriteStream.Writable.write (_stream_writable.js:179:12)

I'm able to do

this.push(chunk)

to pipe the data directly through, but I can't do

var result = {'the web content is': chunk}
this.push(result)

runnable example in 30 LOC:

var stream = require('stream');

var MsgExtractStream = function() {
  stream.Transform.call(this,{objectMode: true});
}

MsgExtractStream.prototype = Object.create(
  stream.Transform.prototype, {constructor: {value: MsgExtractStream}} )

MsgExtractStream.prototype._transform = function(chunk, encoding, callback) {
  var result = {'the website is': chunk};
  this.push(result);
}

MsgExtractStream.prototype.write = function () {
  this._transform.apply(this, arguments);
};

MsgExtractStream.prototype.end = function () {
  this._transform.apply(this, arguments);
  this.emit("end");
};

var fs = require("fs"),
  inPage = fs.createReadStream("inPage.html"),
  outPage = fs.createWriteStream("outPage.html"),
  msgPage = new MsgExtractStream();

inPage.
pipe(msgPage).
pipe(outPage);

Answer 1

Everything you are doing is correct, except the last statement. You are piping the results of MsgExtractStream a Transform stream with objectMode on to a simple stream.

When you pipe, you read stuff from msgPage, you get objects and then write them into outPage. But outPage does not have objectMode on, so it needs strings/buffer and throws the error when you write to it.

Try doing only one of the pipes, it will work.

Edit

That was my method to debug the error, one pipe does not cause error. First we will check the flow of data which is something like this:

string stream -> object stream -> string stream
             string          object

The chunk you get in MsgExtractStream is a string from inPage. It would be OK with any object not just string. In the transform you write chunk wrapped in object to output. You get a string and write an object.

You have two options now. To resolve the problem either MsgExtractStream should return string, like when you do this.push(chunk), so that next pipe works.

Or if you need the object stream, you get from MsgExtractStream. You will need another pipe like this:

string stream -> object stream -> object stream -> string stream
            string           object           string

First object stream creates an object from string, second creates string from an object. You will have to handle the type of objects along the chain, so that it pipes.