How do I scrape data from this webpage with python using BeautifulSoup? I'm specifically trying to get the last table which includes data of all neighborhoods.
You can use this example to load the data:
import requests
import pandas as pd
headers = {
"User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0",
"Accept-Language": "en-US,en;q=0.5",
}
r = requests.get(
"https://www.zolo.ca/toronto-real-estate/neighbourhoods", headers=headers
)
df = pd.read_html(r.text)[-1]
df.iloc[:, 0] = df.iloc[:, 0].str.replace(r"(\d+)(.*)", r"\1 \2")
print(df)
Prints:
Neighbourhood (# Rank out of 143) Sold under 10d Sold above asking Average sale price Active listings
0 19 Agincourt North 72% 73% $951K 25
1 71 Agincourt South-Malvern West 53% 60% $692K 42
2 59 Alderwood 63% 53% $1.3M 30
3 141 Annex 31% 20% $1.5M 129
4 107 Banbury-Don Mills 46% 36% $1.3M 55
5 80 Bathurst Manor 51% 51% $1.3M 16
6 142 Bay Street Corridor 28% 19% $777K 154
7 113 Bayview Village 42% 38% $834K 72
8 66 Bayview Woods-Steeles 60% 48% $1.3M 9
9 91 Bedford Park-Nortown 54% 34% $2.4M 57
10 5 Beechborough-Greenbrook 86% 86% $982K 5
11 64 Bendale 58% 58% $766K 30
12 63 Birchcliffe-Cliffside 58% 59% $1.0M 40
13 112 Black Creek 40% 45% $550K 6
14 10 Blake-Jones 82% 73% $1.3M 6
15 70 Briar Hill-Belgravia 56% 50% $812K 13
16 140 Bridle Path-Sunnybrook-York Mills 31% 21% $4.4M 53
17 42 Broadview North 67% 57% $1.1M 6
18 134 Brookhaven-Amesbury 33% 33% $853K 11
19 83 Cabbagetown-South St. James Town 53% 45% $1.2M 20
20 45 Caledonia-Fairbank 63% 67% $988K 12
21 108 Casa Loma 45% 34% $1.7M 36
22 11 Centennial Scarborough 80% 75% $1.1M 8
23 135 Church-Yonge Corridor 35% 28% $706K 172
24 9 Clairlea-Birchmount 78% 89% $919K 13
25 96 Clanton Park 47% 47% $1.0M 36
26 35 Cliffcrest 68% 66% $1.3M 19
27 47 Corso Italia-Davenport 63% 63% $1.2M 18
28 114 Crescent Town 44% 32% $566K 16
29 8 Danforth 84% 74% $1.3M 6
30 6 Danforth Village-East York 85% 76% $1.3M 16
31 105 Don Valley Village 49% 33% $994K 25
32 73 Dorset Park 51% 60% $731K 20
33 38 Dovercourt-Wallace Emerson-Junction 69% 57% $1.1M 41
34 89 Downsview-Roding-CFB 50% 48% $869K 64
35 30 Dufferin Grove 70% 67% $1.1M 9
36 24 East End-Danforth 73% 67% $1.1M 13
37 7 East York 84% 76% $1.4M 12
38 136 Edenbridge-Humber Valley 37% 22% $1.5M 31
39 51 Eglinton East 60% 71% $756K 14
40 143 Elms-Old Rexdale 15% 30% $561K 10
41 77 Englemount-Lawrence 55% 42% $1.2M 27
42 44 Eringate-Centennial-West Deane 66% 58% $1.1M 15
43 103 Etobicoke West Mall 45% 45% $745K 16
44 132 Flemingdon Park 33% 36% $591K 31
45 68 Forest Hill North 60% 40% $1.8M 9
46 121 Forest Hill South 41% 34% $2.3M 29
47 40 Glenfield-Jane Heights 65% 63% $821K 14
48 20 Greenwood-Coxwell 71% 74% $1.3M 21
49 18 Guildwood 72% 76% $996K 16
50 119 Henry Farm 42% 35% $633K 40
51 81 High Park North 53% 44% $1.2M 16
52 94 High Park-Swansea 48% 47% $1.2M 23
53 27 Highland Creek 73% 64% $1.4M 20
54 16 Hillcrest Village 76% 76% $1.1M 25
55 85 Humber Heights 55% 36% $869K 22
56 87 Humber Summit 50% 50% $794K 9
57 1 Humberlea-Pelmo Park W4 100% 100% $942K 5
58 32 Humberlea-Pelmo Park W5 68% 71% $942K 10
59 95 Humbermede 45% 55% $673K 8
60 53 Humewood-Cedarvale 65% 55% $1.5M 4
61 50 Ionview 57% 79% $824K 7
62 Islington 0% 0% $0 2
63 111 Islington-City Centre West 42% 41% $822K 65
64 34 Junction Area 67% 72% $1.0M 9
65 86 Keelesdale-Eglinton West 48% 57% $886K 13
66 54 Kennedy Park 62% 59% $889K 16
67 139 Kensington-Chinatown 29% 32% $849K 37
68 106 Kingsview Village-The Westway 46% 40% $772K 33
69 118 Kingsway South 42% 35% $2.3M 21
70 37 L'Amoreaux 65% 72% $853K 36
71 23 Lambton Baby Point 71% 71% $1.7M 4
72 131 Lansing-Westgate 35% 31% $1.2M 30
73 61 Lawrence Park North 61% 53% $1.9M 10
74 41 Lawrence Park South 71% 45% $3.1M 31
75 72 Leaside 57% 43% $2.1M 28
76 97 Little Portugal 48% 43% $809K 33
77 69 Long Branch 58% 45% $979K 23
78 58 Malvern 58% 65% $662K 51
79 101 Maple Leaf 47% 41% $1.2M 11
80 98 Markland Wood 48% 41% $850K 10
81 33 Milliken 66% 75% $860K 27
82 99 Mimico 48% 40% $800K 131
83 46 Morningside 59% 76% $757K 10
84 93 Moss Park 50% 43% $731K 53
85 110 Mount Dennis 35% 61% $666K 18
86 128 Mount Olive-Silverstone-Jamestown 33% 39% $557K 21
87 76 Mount Pleasant East 55% 45% $1.4M 34
88 133 Mount Pleasant West 35% 29% $780K 66
89 48 New Toronto 63% 63% $1.2M 11
90 120 Newtonbrook East 45% 25% $1.6M 44
91 88 Newtonbrook West 52% 43% $1.1M 60
92 92 Niagara 49% 44% $754K 78
93 21 North Riverdale 70% 78% $1.8M 7
94 125 North St. James Town 36% 36% $790K 8
95 43 O'Connor-Parkview 67% 57% $1.4M 12
96 36 Oakridge 69% 62% $948K 8
97 39 Oakwood-Vaughan 69% 54% $1.1M 25
98 60 Palmerston-Little Italy 67% 39% $1.3M 9
99 74 Parkwoods-Donalda 52% 55% $951K 29
100 14 Playter Estates-Danforth 80% 70% $1.6M 1
101 67 Pleasant View 56% 60% $907K 11
102 28 Princess-Rosethorn 73% 64% $1.8M 8
103 90 Regent Park 51% 43% $714K 19
104 26 Rexdale-Kipling 70% 73% $916K 9
105 79 Rockcliffe-Smythe 51% 54% $941K 22
106 29 Roncesvalles 72% 65% $1.4M 22
107 124 Rosedale-Moore Park 42% 24% $2.1M 73
108 2 Rouge E10 94% 94% $1.0M 7
109 22 Rouge E11 68% 81% $914K 24
110 3 Runnymede-Bloor West Village 91% 82% $1.6M 7
111 12 Rustic 71% 100% $1.0M 8
112 84 Scarborough Village 47% 61% $930K 28
113 100 South Parkdale 46% 46% $1.0M 5
114 25 South Riverdale 72% 69% $1.1M 52
115 102 St. Andrew-Windfields 48% 36% $2.6M 55
116 49 Steeles 63% 63% $1.1M 24
117 78 Stonegate-Queensway 53% 48% $1.3M 33
118 56 Tam O'Shanter-Sullivan 61% 62% $857K 18
119 15 The Beaches 80% 70% $1.7M 43
120 65 Thistletown-Beaumonde Heights 57% 57% $1.1M 8
121 138 Thorncliffe Park 36% 18% $592K 2
122 104 Trinity-Bellwoods 44% 46% $1.6M 21
123 116 University 44% 29% $1.2M 17
124 31 Victoria Village 71% 63% $940K 11
125 123 Waterfront Communities C1 41% 29% $727K 377
126 130 Waterfront Communities C8 35% 31% $802K 87
127 57 West Hill 60% 65% $849K 35
128 117 West Humber-Clairville 41% 38% $695K 27
129 115 Westminster-Branson 41% 40% $766K 27
130 62 Weston 58% 60% $770K 16
131 75 Weston-Pellam Park 53% 53% $979K 10
132 55 Wexford-Maryvale 62% 60% $990K 21
133 126 Willowdale East 36% 32% $893K 139
134 122 Willowdale West 42% 32% $1.1M 44
135 52 Willowridge-Martingrove-Richview 63% 63% $1.1M 19
136 82 Woburn 50% 54% $746K 36
137 13 Woodbine Corridor 79% 79% $1.2M 4
138 4 Woodbine-Lumsden 86% 86% $1.1M 8
139 17 Wychwood 78% 63% $1.4M 8
140 109 Yonge-Eglinton 44% 38% $1.5M 15
141 137 Yonge-St. Clair 36% 23% $1.6M 33
142 129 York University Heights 34% 33% $833K 23
143 127 Yorkdale-Glen Park 36% 33% $802K 43