urllib.error.HTTPError: HTTP Error 502: Bad Gateway PYTHON

Question

I am trying to open the Instagram URL using urllib.request.urlopen(url).read() but I am getting the error urllib.error.HTTPError: HTTP Error 502: Bad Gateway

username = input('enter the username - ')
url = "https://www.instagram.com/{}".format(username)
html = urllib.request.urlopen(url).read()
soup = BeautifulSoup(html, 'html.parser')

How can I solve this issue?

EDIT: I got the solution and I have posted it. Cheers :)

Answer 1

First, add
req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})
then type
html = urllib.request.urlopen(req).read()
This will solve the problem. Previously it couldn't verify the origin of the request. We tricked it into thinking that the request is made from a genuine browser, i.e. Mozilla.