Search code examples
python-3.xbeautifulsoupyelp

Scraping Yelp restaurants address

I tried to extract the address and the postal code of the restaurants from Yelp but without success. The problem I encountered is that I can't extract the second tag which contains the zip code. The code below returns the address and not the postal code yet in the image below contains 2 thread tags, the first contains the address and the second contains the postal and the city.

from bs4 import BeautifulSoup
import requests

url = 'https://www.yelp.com/search?cflt=restaurants&find_loc=Montreal, QC'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')
for item in soup.select('[class*=container]'):
  try:
    if item.find('h4'):
      name = item.find('h4').get_text()
      addr = item.find('address').get_text()
      print(name)
      print(addr)
      print('------------------')
  except Exception as e:
    raise e
    print('')

Inspect element:

enter image description here

Solution

  • You can try and use find_all

    from bs4 import BeautifulSoup
import requests

url = 'https://www.yelp.com/search?cflt=restaurants&find_loc=Montreal, QC'
response = requests.get(url)
soup = BeautifulSoup(response.content, 'lxml')
for item in soup.select('[class*=container]'):
  try:
    if item.find('h4'):
      name = item.find('h4').get_text()
      print(name)
      for addr in item.find_all('address'):
          print (addr.text, addr.next_sibling.text)
  except Exception as e:
    raise e
    print('')