Search code examples
node.jsstreamnode-streams

Usage of stream.Transform.from


I want to quickly declare a transform stream without any additional libraries. Making async generators into transform streams via stream.Transform.from looks like a good choice.

someReadable.pipe(
        stream.Transform.from(async function* (source, writable) {
          for await (const chunk of source) {
            yield JSON.stringify(chunk, null, 2) + "\n\n";
          }
        })
      )

Why does the above not work?

TypeScript throws:

Error:(8, 9) TS2345: Argument of type 'Readable' is not assignable to parameter of type 'WritableStream'.
  Type 'Readable' is missing the following properties from type 'WritableStream': writable, write, end
Error:(8, 31) TS2345: Argument of type '(source: any, writable: any) => AsyncGenerator<string, void, unknown>' is not assignable to parameter of type 'Iterable<any> | AsyncIterable<any>'.
  Property '[Symbol.asyncIterator]' is missing in type '(source: any, writable: any) => AsyncGenerator<string, void, unknown>' but required in type 'AsyncIterable<any>'.
Error:(14, 8) TS2339: Property 'pipe' does not exist on type 'WritableStream'.

Solution

  • The read function is not actually from the Transform class. Transform is a child class of Duplex which is a child of Readable and Writable. The only .from function in the stream module is the Readable.from function, so you're actually calling that.

    You can verify this yourself:

    $ node
    Welcome to Node.js v12.14.0.
    Type ".help" for more information.
    
    > const stream = require('stream')
    > stream.Readable.from === stream.Transform.from
    true
    

    Unfortunately, the stream module does not appear to have Transform.from or Writable.from.

    From https://nodejs.org/api/stream.html#stream_types_of_streams:

    Additionally, this module includes the utility functions stream.pipeline(), stream.finished() and stream.Readable.from().