Node.js Axios ERR_FR_TOO_MANY_REDIRECTS, but no infinite loop on curl, browser or python-requests

Question

I'm trying to fetch the following url using Axios on NodeJS: https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk

When doing so with Axios, I'm getting ERR_FR_TOO_MANY_REDIRECTS. When looking at the error carefully, to me it seems like it's possibly because of faulty encoding of the redirected URL, as eventually it should result to https://chromewebstore.google.com/detail/%E5%85%AD%E7%9C%81%E5%8F%91%E6%96%87/bleabhffpgoeopcfeapggnbdlhklbajk (with Chinese chars if entered in the browser).

I tried many many things, including using Axios interceptors to reencode the URL, but nothing works.

Surprisingly, everything works when using curl (curl -L https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk) or python requests (requests.get(' https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk').

Here's the code I used, where I tried to encode the redirect URL to try and somehow resolve it (obviously the simpler version doesn't work as well):

    const axiosInstance = axios.create({
        maxRedirects: 5,
        responseType: 'arraybuffer'
    });

    try {
        const response = await axiosInstance.get('https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk', {
            headers: {
                'User-Agent': userAgent,
                'Content-Type': 'text/html; charset=UTF-8',
                'Accept-Language': 'en-US,en;q=0.5'
            }
        });
        return response.data;
    } catch (error) {
        if (error.response && error.response.status === 302) {
            const location = error.response.headers.location;
            console.log('Original redirect URL:', location);

            // Split the URL into its components
            const urlParts = location.split('/');

            // Find the index of the "detail" part
            const detailIndex = urlParts.findIndex(part => part === 'detail');

            if (detailIndex !== -1 && detailIndex + 1 < urlParts.length) {
                // Encode the part after "detail"
                urlParts[detailIndex + 1] = encodeURIComponent(urlParts[detailIndex + 1]);

                // Join the URL parts back together
                const modifiedLocation = urlParts.join('/');
                console.log('Modified redirect URL:', modifiedLocation);

                // Send a new request with the modified URL
                const response = await axiosInstance.get(modifiedLocation, {
                    headers: {
                        'User-Agent': userAgent,
                        'Accept-Charset': 'UTF-8',
                        'Content-Type': 'text/html; charset=UTF-8'
                    }
                });
                return response.data;
            } else {
                throw new Error('Invalid URL format');
            }
        } else {
            throw error;
        }
    }
}

Any idea?

Thanks in advance!

Answer 1

This was a bug in the undici implementation of fetch, which has been fixed with https://github.com/nodejs/undici/pull/2971.

The newest version of Node.js, v22.0.0 does not exhibit this error any more.

>node
Welcome to Node.js v22.0.0.
Type ".help" for more information.
> (await fetch('https://chromewebstore.google.com/detail/bleabhffpgoeopcfeapggnbdlhklbajk')).url
'https://chromewebstore.google.com/detail/%E5%85%AD%E7%9C%81%E5%8F%91%E6%96%87/bleabhffpgoeopcfeapggnbdlhklbajk'