Stream behaviour is different after upgrading from Node 14 to Node 18

Question

I'm new to Node.js and I need to upgrade the node version of a project from Node 14 to Node 18. Almost all things are working but one of the unit tests failed. Here's the unit test:

import { ValidationError } from "../../errors"
import { validateCSVHeader } from "../validateCSVHeader"

describe("validateCSVHeader", () => {
  it("Should not throw error if validHeaders included in buffer", async () => {
    const csvBuffer = Buffer.from(`yo,playerId
    1234567,abcd
    7654321,efgh
    `)

    await expect(
      validateCSVHeader(csvBuffer, { validHeaders: ["accountId", "playerId"] })
    ).resolves.not.toThrowError()
  })

  it("Should pass if no specific headers is required", async () => {
    const csvBuffer = Buffer.from(`yo,playerId
    1234567,abcd
    7654321,efgh
    `)

    await expect(validateCSVHeader(csvBuffer)).rejects.toThrowError()
  })

  it("Should throw error if validHeaders not included in buffer", async () => {
    const csvBuffer = Buffer.from(`yo,playerId
    1234567,abcd
    7654321,efgh

    `)

    await expect(
      validateCSVHeader(csvBuffer, { validHeaders: ["accountId"] })
    ).rejects.toThrow(new ValidationError("Invalid CSV Header"))
  })

  it("Should throw error if csv format", async () => {
    const csvBuffer = Buffer.from("")

    await expect(
      validateCSVHeader(csvBuffer, { validHeaders: ["accountId", "playerId"] })
    ).rejects.toThrow(new ValidationError("Invalid CSV Header"))
  })
})

It gives the following error:

And here's the source code:

import parse from "csv-parse"
import stream, { Readable, Writable } from "stream"
import util from "util"

import { ValidationError } from "../errors"

const pipeline = util.promisify(stream.pipeline)

type ValidationsType = {
  validHeaders?: string[]
}

export const validateCSVHeader = async (
  fileBuffer: Buffer,
  validations?: ValidationsType
) => {
  const { validHeaders = [] } = validations || {}

  const csvParser = parse({
    delimiter: ",",
    bom: true,
    columns: false,
    skip_empty_lines: true,
    to_line: 1,
    record_delimiter: ["\r", "\n", "\r\n"],
  })

  const readable = Readable.from(fileBuffer)

  const data: any[] = []
  const writable = new Writable({
    objectMode: true,
    write(chunk, _encoding, callback) {
      data.push(chunk)
      callback()
    },
  })

  await pipeline(readable, csvParser, writable)

  console.log(data)

  const [firstLine] = data
  if (firstLine && validHeaders.some((header) => firstLine.includes(header))) {
    return
  }

  throw new ValidationError("Invalid CSV Header", {
    fileHeaders: firstLine,
    validHeaders: validHeaders,
  })
}

I have tried adding a readable.resume() call after creating the readable stream but without luck.

When I do console.log(data), it outputs empty [], but it will output ['yo', 'playerId'] in Node 14. May I know what's wrong and how can I update the code? Thank you.

Answer 1

Finally, I found out that the version of the package csv-parse we are using is not supporting Node 18. It's working after upgrading the package csv-parse to 5.5.0