Using find_all in bs4

Question

I am encountering an issue in my Python script while attempting to parse multiple classes using BeautifulSoup. The error message specifically points to line 12, stating, "ResultSet object has no attribute 'find'. You're probably treating a list of elements like a single element."

import requests
from bs4 import BeautifulSoup

heroes_page_list=[]


url = f'https://dota2.fandom.com/wiki/Dota_2_Wiki'
q = requests.get(url)
result = q.content
soup = BeautifulSoup(result, 'lxml')

heroes = soup.find_all('div', class_= 'heroentry').find('a')
for hero in heroes:
    hero_url = heroes.get('href')
    heroes_page_list.append("https://dota2.fandom.com" + hero_url)
# print(heroes_page_list)

with open ('heroes_page_list.txt', "w") as file:
    for line in heroes_page_list:
        file.write(f'{line}\n')

Answer 1

You are searching a tag inside a list of div tags you need to do like this,

heroes = soup.find_all('div', class_= 'heroentry')
a_tags = [hero.find('a') for hero in heroes]
for a_tag in a_tags:
    hero_url = a_tag.get('href')
    heroes_page_list.append("https://dota2.fandom.com" + hero_url)

heroes_page_list look like this,

['https://dota2.fandom.com/wiki/Abaddon',
 'https://dota2.fandom.com/wiki/Alchemist',
 'https://dota2.fandom.com/wiki/Axe',
 'https://dota2.fandom.com/wiki/Beastmaster',
 'https://dota2.fandom.com/wiki/Brewmaster',
 'https://dota2.fandom.com/wiki/Bristleback',
 'https://dota2.fandom.com/wiki/Centaur_Warrunner',
 ....