Download a file using Python without selenium like Chrome's "Save Link As"

Question

There is a web page where I can download zip files using "Save Link As" option in chrome but when I copy link address and open that in the browser it returns 403/forbidden. I tried to save the file using requests library but it also gets forbidden response.

I don't know how chrome can download it but I can't download using requests library.

How can I download the file without using selenium web driver as that will be overkill for this simple task?

Answer 1

I would recommend using requests for this. Simple example below with the first file filled in:

url = 'https://www.nseindia.com/content/historical/EQUITIES/2003/DEC/cm01DEC2003bhav.csv.zip'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36', 'Referer': 'https://www.nseindia.com/'}
r = requests.get(url, allow_redirects=True, headers=headers)
open('cm01DEC2003bhav.csv.zip', 'wb').write(r.content)

The website checks for referer in the header, if referer doesn't match with the website itself it denies the request.