How to prevent the use of urllib2 package in python 2.7?

Question

I am trying to do the same thing below, but without using the urllib2 package. I am trying to web scrape a URL (not exactly the one shown below). Must have headers for security reasons.

URL = 'https://www.google.com/search?q=test'
hdr = {'User-Agent': 'Mozilla/5.4'}
req = urllib2.Request(URL, headers=hdr)
pag = urllib2.urlopen(req)
soup = BeautifulSoup(pag, "lxml")
all_tables = soup.find_all('table')
right_table = soup.find_all('table')[1]

I have tried looking this up in stack overflow, but I can only find solutions using urllib2. I have reasons for not wanting to use urllib2

Is there any way to do this without using urllib2? I'm using python 2.7

Thanks.

Answer 1

import requests

URL = 'https://www.google.com/search?q=test'
hdr = {'User-Agent': 'Mozilla/5.4'}
req = requests.get(URL, headers=hdr)
soup = BeautifulSoup(req.content, "lxml")
all_tables = soup.find_all('table')
right_table = soup.find_all('table')[1]