Search code examples
node.jsnode-fetch

Why Can't I Fetch a Webpage (With NodeJS and Node-Fetch)?


I am trying to fetch a site: link here. If you click on the link, it shows JSON: {"error":"Socket Error"}. I am trying to fetch that website, and return the error.

However, I get a 403 Forbidden error instead. Is there a reason for this? I turned CORS off, but I don't think it did anything. Here is an example of what I have tried:

async function b(){
  error = await fetch('https://matchmaker.krunker.io/seek-game?hostname=krunker.io&region=us-ca-sv&game=SV%3A4jve9&autoChangeGame=false&validationToken=QR6beUGVKUKkzwIsKhbKXyaJaZtKmPN8Rwgykea5l5FkES04b6h1RHuBkaUMFnu%2B&dataQuery=%7B%7D', {mode:'no-cors'}).then(res=>res.json())
  console.log(JSON.stringify(error))
}
b()

Why doesn't anything seem to work? Please comment if there is anything I need to add, this is my first Stack Overflow post so I am still slightly confused by what makes a good question. Thanks for helping!!

NOTE: My environment is Node.JS (testing on Repl.it which I think uses the latest Node version).


Solution

  • This particular host is protected width Cloudflare anti DDoS protection. The server doesn't accept requests made by fetch, but the do accept requests from curl. God knows why.

    $ curl 'https://matchmaker.krunker.io/seek-game?hostname=krunker.io&region=us-ca-sv&game=SV%3A4jve9&autoChangeGame=false&validationToken=QR6beUGVKUKkzwIsKhbKXyaJaZtKmPN8Rwgykea5l5FkES04b6h1RHuBkaUMFnu%2B&dataQuery=%7B%7D'
    
    // => {"error":"Socket Error"}
    

    You can use curl in node.js with node-libcurl package.

    const { curly } = require('node-libcurl')
    const url = 'https://matchmaker.krunker.io/seek-game?hostname=krunker.io&region=us-ca-sv&game=SV%3A4jve9&autoChangeGame=false&validationToken=QR6beUGVKUKkzwIsKhbKXyaJaZtKmPN8Rwgykea5l5FkES04b6h1RHuBkaUMFnu%2B&dataQuery=%7B%7D'
    
    curly.get(url)
        .then(({ statusCode, data }) => console.log(statusCode, data))
    
    // => 400 { error: 'Socket Error' }
    

    Works as expected :-)