Puppeteer fails to navigate to url (ERR_EMPTY_RESPONSE)

The problem This is the simplest code you can write to navigate to a page using puppeteer:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  page.on('response', response => {console.log(response.request().url())});
  page.on('error', error => {console.error(error.message)});
  await page.goto('https://www.vueling.com/es');
  await browser.close();
})();

But this generates an error:

UnhandledPromiseRejectionWarning: Error: net::ERR_EMPTY_RESPONSE at https://www.vueling.com/es

The question

This page loads perfectly when I try myself in Chrome/Chromium (~135 requests in 5 ~7"). My question is, is this a puppeteers thing, is Chrome's fault or is there something else I'm missing? Why does this not work?

Environment

Puppeteer version: 1.10.0
Platform / OS version: macOS High Sierra 10.13.6
Node.js version: 10.13

Solution

Some websites may be detecting puppeteer because it has a particular user-agent : Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/72.0.3617.0 Safari/537.36. You can notice the Headless Chrome in this one.

If you override your user-agent : await page.setUserAgent("Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"); before loading the page with a regular browser user-agent, it works as expected.

If a website tries to detect puppeteer (there are plenty of other ways to do it), it is because they don't want their information to be accessed automatically. So if you run it on a website that you don't own, you should respect its data.