Search code examples
javascriptnode.jssquarespacenode-request

Node.js can't request a squarespace site?

I'm trying to request this website but I keep getting a 400 Bad Request error. This code works for just about any other site I've tried that isn't built with squarespace so I'm guessing that's where the problem is. 

var request = require('request');
var cheerio = require('cheerio');
var url = 'http://www.pond-mag.com/';



request(url, function(error, resp, body){
    if(!error){
        var $ = cheerio.load(body);
        console.log(body);
    }
});
Solution

  • Figured it out just had to manually set the headers object. Heres the code that fixed it in case anyone else has the problem:

    var options = {
url : 'http://www.pond-mag.com/',
  headers: {
   'User-Agent': 'request'
  }
};

    Then, just pass the options var to the request instead of the url.