Search code examples
pythonexport-to-csv

How to export dict list to csv cleanly?


I have a script I'm writing to make pulling data from my fantasy football league easy and exported in a format that can be played with in Excel easily.

The script I have attached only contains the relevant parts to this questions as the larger script I have written has a lot of moving parts that doesn't apply here.

I'm essentially pulling this players.get_all_players() data from the Sleeper platform using the Sleeper-API-Wrapper (Github link here).

My script will take player data and put it into a .csv like this, with the player ID in the top row and all the info in a single cell below the ID. Screenshot of this below.

Excel .csv screenshot

How can I export this so that the data is nicely formatted into separate rows? I have a different spreadsheet where I'd like to be able to pull this data to automatically.

Alternatively, if I'm doing this in a really roundabout way, please let me know! This is the JSON response from the platform: JSON Response

# 9 All players - players.get_all_players()

warning = 1
while warning == 1:
    print("%s%s\n\n\nWARNING:%s" % (fg(15), bg(9), attr(0)))
    print("%s%sthe 'all players' option is intensive and may freeze your PC for several minutes.%s" % (fg(15), bg(0), attr(1)))
    warning = input("continue anyway? (y/n)\n")
    if warning == "n":
        pe_loop = 0
        action = 0
    elif warning == "y":
        name = "all players"; file = name
        output = players.get_all_players()
        break
    else:
        print("Not a valid option, try again.")
        warning = 1

overwrite = 0
name_change = 0
while action == 0:
    try:
        action = int(input("%s%s\n1 - print\n2 - export to Excel\n3 - back to tasks\n4 - end\n--------------------\n%s" % (fg(14), bg(0), attr(1))))
    except ValueError:
        print("Not a valid option, try again.")

## Print
    if action == 1 and week != 18:
        print(output)
        break

    elif action == 1 and week == 18:
        week = 0
        while week < 18:
            week += 1
            if task == 3:
                output = league.get_matchups(week)
            elif task == 4:
                output = league.get_transactions(week)
            print(output)

## Export
    elif action == 2:
        path = os.path.join(parent_dir, file)
        name_change = input("\nDo you want to name the file? (y/n)\n")
        if name_change == "y":
            name = input("\nEnter file name now:\n")
        if name_change == "n":
            file_path = path + "\\" + name + '_' + str(year) + ".xlsx"
            if os.path.isfile(file_path) == True:
                overwrite = input("\nFile name... '" + name + "' already exists! Would you like to overwrite this file? (y/n)\n")
                if overwrite == "n":
                    count = 0
                    while os.path.isfile(file_path) == True:
                        count += 1
                        new_name = name + "_" + str(count)
                        file_path = path + "\\" + new_name + ".xlsx"

                    else:
                        name = new_name
                    
                    print("\nThe new file was automatically named: " + new_name + "_wk" + str(week) + "\nand placed in: " + path)
                    
        if os.path.isdir(path) == False and overwrite == 0:
            os.mkdir(path)
            print("\nCreating new file path... " + file + "\n")
        elif os.path.isdir(path) == True and overwrite == 0:
            print("\nFile path... '" + file + "' exists!\n")
        toCSV = output

# 9 All Players CSV file
        with open(parent_dir + file + "\\" + name + ".csv", 'w', encoding='utf8', newline='') as output_file:
            fc = csv.DictWriter(output_file, output.keys())
            fc.writeheader()
            fc.writerow(toCSV)

Solution

  • It turns out that sleeper_wrapper exposes a method players.get_players_df that gives you a pandas DataFrame containing all players.

    Write that to a csv file using to_csv as suggested in the comments.

    Strip down your code to receive better answers faster :) This is the code that your question needs:

    from sleeper_wrapper import Players
    import csv
    
    players = Players()
    toCSV = players.get_all_players()
    
    with open(parent_dir + file + "\\" + name + ".csv", 'w', encoding='utf8', newline='') as output_file:
                fc = csv.DictWriter(output_file, output.keys())
                fc.writeheader()
                fc.writerow(toCSV)
    

    This is how you write the csv using pandas:

    import pandas as pd
    from sleeper_wrapper import Players
    
    players = Players()
    all_players = players.get_all_players()
    # stolen from https://github.com/NotTheCrocHunter/sleeper-api-wrapper/blob/91d8cf1b64cf55884b4c4746d53ccd1259d11c1f/sleeper_wrapper/players.py#L41
    # because that method is unavailable in the version of sleeper_wrapper in PyPI
    all_players_df = pd.DataFrame.from_dict(all_players, orient="index")
    # all_players_df contains some information on teams as well, maybe you want to filter that out...
    all_players_df.to_csv("your_output_file.csv")