Search code examples
pythonarraysappendconcatenationscreen-scraping

Python - Concatenate two for loops into array


I am new to python and learning on the fly, trying to combine knowledge from multiple tutorials, to solve my problem.

Essentially, I am looking to scrape the below website to pull out all the properties and their corresponding page links into an array.

Website : "https://www.accommodationforstudents.com/search-results?location=London&area=&beds=0&searchType=halls&price=undefined&limit=99"

The issue is, when I run the code, it iterates through the links for each property correctly, but the name of the property does not. I would be grateful for any help.

Regards

..........

import urllib.request
import requests
from bs4 import BeautifulSoup

url = "https://www.accommodationforstudents.com/search-results?location=London&area=&beds=0&searchType=halls&price=undefined&limit=99"

response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")

page = soup.findAll('a', attrs={'class': "student-halls-card__link"})
property = soup.findAll('strong', attrs={'class': "student-halls-card__title"})

sites = []


for link in page:
    link.find('href', attrs={'class': "student-halls-card__link"})

    for name in property:
        name.find('href', attrs={'class': 'student-halls-card__title'})

    sites.append(name.text + " - " + "https://www.accommodationforstudents.com" + link.get('href'))

print(sites)

.............

Result shortened.. 'Felda House - https://www.accommodationforstudents.com/student-hall/407', 'Felda House - https://www.accommodationforstudents.com/student-hall/1672', 'Felda House - https://www.accommodationforstudents.com/student-hall/3260', ,................]


Solution

  • Try this.

    for link, name in zip(page, property):