Search code examples
node.jshttprequesthttp-status-code-407

NodeJs HTTP Request Not working with proxy (407)

I've been using the NPM Request package for years now and despite the fact that it's deprecated, it has always worked perfectly for my needs and I haven't ever had any issues up until today...

I am trying to do a regular GET request with a user:pass authenticated proxy and some additional headers - the same thing I've done a thousand times in the past, however this time the url is HTTP instead of HTTPS.

For whatever reason because the link is HTTP it is messing with my proxy authentication and for whatever reason is returning a response code of 407 and the response body shows an error stating Cache Access Denied (ERR_CACHE_ACCESS_DENIED). This is where I'm confused as to what to do because I know for a fact that there shouldn't be anything wrong with my proxy authentication since its the same thing I've done for years and years.

Request Code:

const request = require('request').defaults({
    timeout: 30000,
    gzip: true,
    forever: true
});

cookieJar = request.jar();
proxyUrl = "http://proxyUsername:proxyPassword@proxyDomain:proxyPort";

request({
    method: "GET",
    url: "http://mylink.com",
    proxy: proxyUrl,
    jar: cookieJar,
    headers: {
      "Proxy-Authorization": new Buffer('proxyUsername:proxyPassword').toString('base64'),
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
      "Accept-Language": "en-GB,en-US;q=0.9,en;q=0.8",
      "Connection": "keep-alive",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36",
    },
    followAllRedirects: true
}, (err, resp, body) => {

    if (err || resp.statusCode != 200) {
        if (err) {
            console.log(err);
        } else {
            console.log(resp.statusCode);
            console.log(resp.body);

        }
        return;
    }

    console.log(resp.body);
});

Snippet of Response Body (Status Code 407):

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html><head>
<meta type="copyright" content="Copyright (C) 1996-2019 The Squid Software Foundation and contributors">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>ERROR: Cache Access Denied</title>
<style type="text/css"><!-- 

body
:lang(fa) { direction: rtl; font-size: 100%; font-family: Tahoma, Roya, sans-serif; float: right; }
:lang(he) { direction: rtl; }
 --></style>
</head><body id=ERR_CACHE_ACCESS_DENIED>
<div id="titles">
<h1>ERROR</h1>
<h2>Cache Access Denied.</h2>
</div>
<hr>
...
<p>Sorry, you are not currently allowed to request http://mylink.com from this cache until you have authenticated yourself.</p>
...

Other things to note:

  • I have also connected to the exact same proxy on my browser and gone to the same site and it works perfectly so it's not an issue with the proxy itself.
  • If I remove the proxy from the request it works perfectly, so maybe I have configured the request wrong for HTTP is all I can think of

Now as I said this exact code works flawlessly with any HTTPS link so that's where I'm stumped. Any help would be appreciated!

Solution

  • Changed the Proxy-Authorization header to Proxy-Authenticate and that seemed to work perfectly. I have no clue why it requires that for HTTP but there you go, not much documentation on the matter...