I am trying to scrape the website for each teams roster and then export it into a csv file. The code is able to scrape the data but it only saves the final teams data (New York Rangers) into the csv. Just wondering how I can save all the scraped data from the loop?
import pandas as pd
import time
team = ['Toronto Maple Leafs', 'Montreal Canadiens', 'Boston Bruins', 'Chicago Blackhawks', 'Detroit Red Wings', 'New York Rangers']
code = ['TOR', 'MTL', 'BOS', 'CHI', 'DET', 'NYR']
for i in range(0, len(code)):
codes = code[i]
teams = team[i]
df = pd.read_html(f'https://www.hockey-reference.com/teams/{codes}/#roster', match='Roster')
df = df[0]
df = df[['Player']]
df = df.assign(Team=[teams]*len(df), Rating=['99']*len(df))
print(df)
time.sleep(5)
df.to_csv('Players.csv')
print("Saved!")
You can do mode='a' to append all the new data to the existing data in the loop.
df.to_csv('Players.csv', mode='a')