How to increment header size in a node application

I'm creating a node application to read a csv file that has 50k lines. Each line consists of a counter, a slug and a category. From this file I get the slug and call with fetch the URL and get a json object back. Using fs to read the file and return it as an object. Just getting index and slug back.

after that I'm using this for to iterate through the object.

for (const [index, { slug }] of slugs.entries())

In the for I'm basically just doing the api fetch

    const apiUrl = `${BASE_URL}/${slug}`;

    const response = await fetch(apiUrl);
    const data = await response.json();

And processing the data with other functions to the point that I have the data ready to post to another api.

Like I mentioned I need to do this for 50k-250k slugs but I get the following error after 60.

FetchError: request to https://api.com/article/best-internet-providers failed, reason: Parse Error: Header overflow

Is there a way of incrementing the header size? I have 64gb ram available to run this program. Also running windows 11.

Solution

To accept larger headers, use --max-http-header-size on the command line. If you get the error:

Node: bad option: --max-http-header-size

Please upgrade to Node v10.15.0 or higher