Search code examples
pythonparsingweb-scrapingbeautifulsouphtml-parsing

Taking multiple prices on single page BS4


I'm creating an to help me learn but is also useful to me. I want to be able to parse multiple prices from (https://www.watchfinder.co.uk/search?q=114060&orderby=AgeNewToOld) one page, convert them to numbers and average them. The page will change so it could have 3 prices one day and 20 the next. The part i am struggling with is separating the prices so that i can use them. So far i have:

page = requests.get(URL, headers=headers)

soup = BeautifulSoup(page.content, 'html.parser')


price = soup.find_all(class_=('prods_price'))
for price in price:
    price = price.text
    price = " ".join(price.split())
    price = price.split('£')
    price = [y.replace(',', '') for y in price]
    price = list(map(int, price[1:]))
    print(price)

Which gives me

[9450]
[8750]
[8450]

Baring in mind that the amount of prices can change, how can I separate these? Or is there a way with BS4 that can get all these without forlooping?


Solution

  • This will provide the average value for all prices,

    URL = 'https://www.watchfinder.co.uk/search?q=114060&orderby=AgeNewToOld'
    page = requests.get(URL)
    soup = BeautifulSoup(page.content, 'html.parser')
    
    prices = soup.find_all(class_=('prods_price'))
    price_list  = [int((price.text).replace('£', '').replace(',', '')) for price in prices]
    print(price_list)
    
    def Average(lst): 
        return sum(lst) / len(lst)
    
    print(Average(price_list))
    

    output:

    [9250, 8750, 8450]

    8816.666666666666