How to receive stream data in NestJS endpoint

Question

I'm attempting to create a micro-service that streams data to another core service when a request is made to it.

Initially, in-order to deal with a large JSON payload I used the following:

@Get('/zzz/stream')
async streamTiles(
  @Param('filterId', new ParseIntPipe()) filterId: number,
  @Query() query: Record<string, string> = {},
  @Res() res: Response
) {
  //... omitted, stream is mongo data stream

  await new Promise((resolve, reject) => {

    stream
      .pipe(JSONStream.stringify())
      .pipe(res)
      .on('finish', resolve)
      .on('error', reject);
  });

This solution worked, but its not as optimal as it could be. It returns 100 rows of extremely large JSON objects.

I'd like to stream from the client side to keep the request brisk & responsive.

However, when attempting the following code:

@Get('/zzz/stream')
async streamTiles(
  @Param('filterId', new ParseIntPipe()) filterId: number,
  @Query() query: Record<string, string> = {},
  @Res({ passthrough: true }) res: Response
) {
   //... omitted, stream is mongo data stream

   res.setHeader('Content-Type', 'application/json');
   res.setHeader('Transfer-Encoding', 'chunked');
   res.setHeader('X-Content-Type-Options', 'nosniff');

   stream.pipe(res);

I receive the following response:

curl: (56) Recv failure: Connection reset by peer

The stream is effectively a MongoDB stream of data from the cursor, and can confirm it is a valid stream.

How do I rectify this issue where the response is appearing empty from curl?

Answer 1

What the error is implying is NestJS has helped you to close the connection forcefully so curl is telling you the connection has been closed.

In your case, the @Res response decorator shouldn't have the value set to { passthrough: true }. When passthrough is set to true, NextJS will auto handle the request for you upon the completion of the function / handler call.

You can remove the { passthrough: true } so that you have full control over the response and it should work as expected.

Excerpt from the documentation

Nest detects when the handler is using either @Res() or @Next(), indicating you have chosen the library-specific option. If both approaches are used at the same time, the Standard approach is automatically disabled for this single route and will no longer work as expected. To use both approaches at the same time (for example, by injecting the response object to only set cookies/headers but still leave the rest to the framework), you must set the passthrough option to true in the @Res({ passthrough: true }) decorator.

Source: https://docs.nestjs.com/controllers