I'm creating my first web scraping application that collects the titles of games currently on the "new and trending" tab on https://store.steampowered.com/. Once I figure out how to do this, I want to repeat the process with prices, and export both to a spreadsheet in separate columns.
I've successfully found the tags that contain the text I'm trying to extract (the title), but I'm unsure how to extract the titles once I've located their containers.
from urllib.request import urlopen
from bs4 import BeautifulSoup
my_url = 'https://store.steampowered.com/'
uClient = urlopen(my_url)
page_html = uClient.read()
uClient.close()
page_soup = BeautifulSoup(page_html, "html.parser")
containers = page_soup.findAll("div",{"class":"tab_item_name"}, limit=10)
for titles in containers:
print(titles)
What I'm trying to do is print the names of the 10 games that appear on steam's homepage in a vertical list using a for loop. What actually happens is I print out the tags that contain the titles:
<div class="tab_item_name">Destiny 2: Shadowkeep</div>
<div class="tab_item_name">Destiny 2</div>
<div class="tab_item_name">Destiny 2: Forsaken</div>
<div class="tab_item_name">Destiny 2: Shadowkeep Digital Deluxe Edition</div>
<div class="tab_item_name">NGU IDLE</div>
<div class="tab_item_name">Kaede the Eliminator / Eliminator 小枫</div>
<div class="tab_item_name">Spaceland</div>
<div class="tab_item_name">Cube World</div>
<div class="tab_item_name">Aokana - Four Rhythms Across the Blue</div>
<div class="tab_item_name">CODE VEIN</div>
Just read the documentation:
If you only want the text part of a document or tag, you can use the get_text() method. It returns all the text in a document or beneath a tag, as a single Unicode string.
So just do:
# Should be `title` IMO, because you are currently handling a single title
for titles in containers:
print(titles.get_text())