Search code examples
pythonjsonperformancememory-efficient

Faster way to interact with JSON with python


I have just started practicing with JSON files and interacting with APIs, in this case, I am interacting with a game's API (Riot API). I have a JSON with my match history, it looks like this (I just copied 3 games, the real file is longer):

{
    "matches": [
        {
            "platformId": "EUW1",
            "gameId": 4733037811,
            "champion": 111,
            "queue": 420,
            "season": 13,
            "timestamp": 1596031907965,
            "role": "DUO_SUPPORT",
            "lane": "BOTTOM"
        },
        {
            "platformId": "EUW1",
            "gameId": 4732867446,
            "champion": 25,
            "queue": 420,
            "season": 13,
            "timestamp": 1596022617021,
            "role": "DUO_SUPPORT",
            "lane": "BOTTOM"
        },
        {
            "platformId": "EUW1",
            "gameId": 4732833555,
            "champion": 111,
            "queue": 420,
            "season": 13,
            "timestamp": 1596020420595,
            "role": "DUO_SUPPORT",
            "lane": "BOTTOM"
        },
        {
            "platformId": "EUW1",
            "gameId": 4731796980,
            "champion": 53,
            "queue": 420,
            "season": 13,
            "timestamp": 1595967793994,
            "role": "DUO_SUPPORT",
            "lane": "BOTTOM"
        },

However, I want more information about my games to do so, I need to search for a specific game with the game. The JSON with the information of a specific game looks like this (I removed some unnecessary stats of each player due to the size of the JSON):

{
    "gameId": 4733037811,
    "platformId": "EUW1",
    "gameCreation": 1596031907965,
    "gameDuration": 1624,
    "queueId": 420,
    "mapId": 11,
    "seasonId": 13,
    "gameVersion": "10.15.330.344",
    "gameMode": "CLASSIC",
    "gameType": "MATCHED_GAME",
    "participants": [
        {
            "participantId": 1,
            "teamId": 100,
            "championId": 61,
            "stats": {
                "participantId": 1,
                "win": true,
                "kills": 14,
                "deaths": 0,
                "assists": 7,
                "totalDamageDealt": 167616,
                "magicDamageDealt": 149177,
                "physicalDamageDealt": 10740,
                "trueDamageDealt": 7698,
                "totalDamageDealtToChampions": 22933,
                "magicDamageDealtToChampions": 21067,
                "physicalDamageDealtToChampions": 1208,
        },
        {
            "participantId": 2,
            "teamId": 100,
            "championId": 21,
            "stats": {
                "participantId": 2,
                "win": true,
                "kills": 2,
                "deaths": 3,
                "assists": 6,
                "totalHeal": 1742,
                "totalUnitsHealed": 2,
                "damageSelfMitigated": 7396,
                "damageDealtToObjectives": 19128,
                "damageDealtToTurrets": 4817,
        },
        {
            "participantId": 3,
            "teamId": 100,
            "championId": 432,
            "stats": {
                "participantId": 3,
                "win": true,
                "kills": 1,
                "deaths": 4,
                "assists": 14,
                "largestKillingSpree": 0,
                "trueDamageDealt": 3202,
                "largestCriticalStrike": 0,
                "totalDamageDealtToChampions": 7395,
                "magicDamageDealtToChampions": 5286,
        {
            "participantId": 4,
            "teamId": 100,
            "championId": 122,
            "stats": {
                "participantId": 4,
                "win": true,
                "kills": 7,
                "deaths": 2,
                "assists": 6,
                "largestKillingSpree": 5,
                "largestMultiKill": 2,
                "killingSprees": 1,
        {
            "participantId": 5,
            "teamId": 100,
            "championId": 79,
            "stats": {
                "participantId": 5,
                "win": true,
                "kills": 1,
                "deaths": 3,
                "assists": 9,
                "largestKillingSpree": 0,
                "largestMultiKill": 1,
                "killingSprees": 0,
                "longestTimeSpentLiving": 422,
                "totalDamageDealt": 143803,
                "magicDamageDealt": 104935,
                "physicalDamageDealt": 31981,
                "trueDamageDealt": 6886,
        },
        {
            "participantId": 6,
            "teamId": 200,
            "championId": 111,
            "stats": {
                "participantId": 6,
                "win": false,
               "kills": 1,
                "deaths": 5,
                "assists": 7,
                "largestKillingSpree": 0,
                "largestMultiKill": 1,
                "longestTimeSpentLiving": 460,
                "totalDamageDealt": 26611,
                "magicDamageDealt": 9267,
                "physicalDamageDealt": 11730,
                "trueDamageDealt": 5613,
                "largestCriticalStrike": 0,
        },
        {
            "participantId": 7,
            "teamId": 200,
            "championId": 56,
            "stats": {
                "participantId": 7,
                "win": false,
                "kills": 3,
                "deaths": 2,
                "assists": 3,
                "totalDamageDealt": 140699,
                "magicDamageDealt": 5378,
                "physicalDamageDealt": 131565,
                "trueDamageDealt": 3755,
                "largestCriticalStrike": 0,
                "totalDamageDealtToChampions": 6557,
                "magicDamageDealtToChampions": 455,
                "physicalDamageDealtToChampions": 5549,
                "trueDamageDealtToChampions": 552,
                "totalHeal": 12548,
                "totalUnitsHealed": 1,
                "damageSelfMitigated": 13095,
                "damageDealtToObjectives": 8279,
                "damageDealtToTurrets": 791,
        },
        {
            "participantId": 8,
            "teamId": 200,
            "championId": 55,
            "stats": {
                "participantId": 8,
                "win": false,
                "kills": 3,
                "deaths": 5,
                "assists": 1,
                "largestKillingSpree": 2,
                "largestMultiKill": 2,
                "killingSprees": 1,
                "longestTimeSpentLiving": 599,
                "doubleKills": 1,
                "tripleKills": 0,
                "quadraKills": 0,
                "pentaKills": 0,
                "unrealKills": 0,
                "totalDamageDealt": 98182,
                "magicDamageDealt": 82014,
                "physicalDamageDealt": 15485,
                "trueDamageDealt": 682,
                "largestCriticalStrike": 0,
                "totalDamageDealtToChampions": 8542,
                "magicDamageDealtToChampions": 6674,
                "physicalDamageDealtToChampions": 1185,
                "trueDamageDealtToChampions": 682,
        },
        {
            "participantId": 9,
            "teamId": 200,
            "championId": 429,
            "stats": {
                "participantId": 9,
                "win": false,
                "kills": 4,
                "deaths": 9,
                "assists": 3,
                "longestTimeSpentLiving": 489,
                "totalDamageDealt": 83886,
                "magicDamageDealt": 2860,
                "physicalDamageDealt": 77851,
                "trueDamageDealt": 3175,
                "largestCriticalStrike": 290,
                "totalDamageDealtToChampions": 16216,
                "magicDamageDealtToChampions": 1500,
                "physicalDamageDealtToChampions": 14715,
                "trueDamageDealtToChampions": 0,
        },
        {
            "participantId": 10,
            "teamId": 200,
            "championId": 68,
            "stats": {
                "participantId": 10,
                "win": false,
                "kills": 1,
                "deaths": 4,
                "assists": 0,
                "totalDamageDealt": 88848,
                "magicDamageDealt": 73156,
                "physicalDamageDealt": 11816,
                "trueDamageDealt": 3875,
                "largestCriticalStrike": 0,
                "totalDamageDealtToChampions": 12531,
                "magicDamageDealtToChampions": 11890,
                "physicalDamageDealtToChampions": 640,
                "trueDamageDealtToChampions": 0,
            },
        }
    ],
}

As you may see, in the JSON file that shows detailed information about the match, it shows the stats for each player. In order to get my information, I looped through each player until reaching the one who played my champion in that game which is shown in the JSON file with my past games.

Therefore, the entire process I did was: loop through each match keeping the value of "gameId" and "champion", then I searched the JSON file with the gameId and looped through all the players of this JSON until finding the one with my "champion" in order to take the information about how many kills, deaths... I had in that game. This is my code:

URL = "https://" + server + ".api.riotgames.com/lol/match/v4/matchlists/by-account/" + account_id + "?api_key=" + ApiKey
response = requests.get(URL)
old_games_json = response.json()
old_games_json = old_games_json['matches']

for index in range(len(old_games_json)):
    champion = (old_games_json[index]['champion'])
    
    gameid = (old_games_json[index]['gameId'])
    
    URL = "https://" + server + ".api.riotgames.com/lol/match/v4/matches/" + str(gameid) + "?api_key=" + ApiKey
    
    response = requests.get(URL)
    game_json = response.json()
    
    for player in range(len(game_json['participants'])):
        if (game_json['participants'][player]['championId']) == champion:
            win = game_json['participants'][player]['stats']["win"]
            kills = game_json['participants'][player]['stats']["kills"]
            deaths = game_json['participants'][player]['stats']["deaths"]
            assists = game_json['participants'][player]['stats']["assists"]

The problem is that I am looping inside a loop which takes a nice amount of time in running the script. My question is that is there any way to make the process faster?

Thanks for taking the time to read this post.


Solution

  • You can build a dictionary champion_stats using a for-comprehension, as shown below (in linear time). The main loop would just be linear time after that (shown at bottom). Note the data has been simplified.

    old_games_json = {
        "matches": [
            {
                "platformId": "EUW1",
                "gameId": 4733037811,
                "champion": 111,
            },
            {
                "platformId": "EUW1",
                "gameId": 4732867446,
                "champion": 25
            },
            {
                "platformId": "EUW1",
                "gameId": 4731796980,
                "champion": 53
            }
        ]
    }
    
    game_json = {
        "gameId": 4733037811,
        "platformId": "EUW1",
        "gameCreation": 1596031907965,
        "gameDuration": 1624,
        "queueId": 420,
        "mapId": 11,
        "seasonId": 13,
        "gameVersion": "10.15.330.344",
        "gameMode": "CLASSIC",
        "gameType": "MATCHED_GAME",
        "participants": [
            {
                "participantId": 1,
                "teamId": 100,
                "championId": 111,
                "stats": {"win": 11}
            },
            {
                "participantId": 2,
                "teamId": 100,
                "championId": 53,
                "stats": {"win": 131}
            },
            {
                "participantId": 3,
                "teamId": 100,
                "championId": 25,
                "stats": {"win": 211}
            }
        ]
    }
    
    champion_stats = dict((participant['championId'], participant['stats']) for participant in game_json['participants'])
    
    for match in old_games_json['matches']:
        champion_id = match['champion']
        stats = champion_stats[champion_id]
        # do stuff here with stats
        print(stats)
    
    

    Output

    {'win': 11}
    {'win': 211}
    {'win': 131}