Search code examples
pythonweb-scrapingwebbeautifulsoup

Beautiful Soup Web Scraping - href

I have the following code:

I want to extract the "href" bit from the html (e.g. the web link: https://storelocator.homebargains.co.uk/store/A779/Quedgeley+Retail+Park,+Gloucester) in this example. Any idea how I'd grab that?

import requests
from bs4 import BeautifulSoup

url = "https://storelocator.homebargains.co.uk/all-stores"
soup = BeautifulSoup(requests.get(url).text, "html.parser")

info = soup.find("td")

print(info)
Solution

  • You could use css selectors to get all the links to the stores avoiding duplicates by selecting them specific:

    ['https://storelocator.homebargains.co.uk'+a.get('href') for a in soup.select('tr td:first-of-type.store a')]

    or use a set comprehension:

    set('https://storelocator.homebargains.co.uk'+a.get('href') for a in soup.select('tr td.store a'))

    To extract the href you could use get('href').

    Example
    import requests
from bs4 import BeautifulSoup

url = "https://storelocator.homebargains.co.uk/all-stores"
soup = BeautifulSoup(requests.get(url).text, "html.parser")

['https://storelocator.homebargains.co.uk'+a.get('href') for a in soup.select('tr td:first-of-type.store a')]
    Output
    ['https://storelocator.homebargains.co.uk/store/A779/Quedgeley+Retail+Park,+Gloucester',
 'https://storelocator.homebargains.co.uk/store/A794/Wren+Retail+Park,+Torquay;+Torquay',
 'https://storelocator.homebargains.co.uk/store/A816/Blairgowrie',
 'https://storelocator.homebargains.co.uk/store/A270/Boulevard+Retail+Park,+Aberdeen',
 'https://storelocator.homebargains.co.uk/store/A277/Inverurie+Retail+Park,+Oldeldrum+Road',
 'https://storelocator.homebargains.co.uk/store/A708/Berryden+Retail+Park,+Aberdeen',
 'https://storelocator.homebargains.co.uk/store/A616/Bridge+of+Don+Retail+Park,+Denmore+Road,+Bridge+of+Don',
 'https://storelocator.homebargains.co.uk/store/A433/Westhill+Shopping+Centre,+Aberdeen',
 'https://storelocator.homebargains.co.uk/store/A131/Eastgate+Retail+Park,+Accrington',
 'https://storelocator.homebargains.co.uk/store/A349/Graham+Street,+Airdrie',
 'https://storelocator.homebargains.co.uk/store/A128/Rookery+Parade,+Aldridge,+West+Midlands',
 'https://storelocator.homebargains.co.uk/store/A136/Institute+Lane,+Alfreton',...]