Search code examples
pythonjsonpandaspython-requestsjson-normalize

How do I handle JSON data obtained from request library in Python?


Im relatively new to using API keys. I've tried IG Markets API and obtain the below. By using r.json(), i've managed to create the type(dictionary). I'm wondering how to handle this type of data and access the values in a "nicer" way.

I've tried using: for key, value in data() : print (key, value) and data["positions"] but it seems that the dictionary only contain one key. Any help is appreciated!

data = r.json()
print(data)
{'positions': [{'market': {'bid': 1719.59,
                           'delayTime': 0,
                           'epic': '-',
                           'expiry': '-',
                           'high': 1749.95,
                           'instrumentName': 'Sverige30 Cash (100SK)',
                           'instrumentType': 'INDICES',
                           'lotSize': 100.0,
                           'low': 1713.33,
                           'marketStatus': 'TRADEABLE',
                           'netChange': -25.44,
                           'offer': 1721.09,
                           'percentageChange': -1.46,
                           'scalingFactor': 1,
                           'streamingPricesAvailable': True,
                           'updateTime': '22:15:16',
                           'updateTimeUTC': '21:15:16'},
                'position': {'contractSize': 100.0,
                             'controlledRisk': False,
                             'createdDate': '2020/07/13 23:12:53:000',
                             'createdDateUTC': '2020-07-13T21:12:53',
                             'currency': 'USD',
                             'dealId': '',
                             'dealReference': '',
                             'direction': 'BUY',
                             'level': 1720.74,
                             'limitLevel': None,
                             'limitedRiskPremium': None,
                             'size': 5.0,
                             'stopLevel': None,
                             'trailingStep': None,
                             'trailingStopDistance': None}}]}

Solution

    • In the example, there's only one record, so I've created an example with multiple records in positions
    • I would use pandas to access all of the data
    import pandas as pd
    
    # create dataframe
    df = pd.json_normalize(data, 'positions')
    
    # display(df)
       market.bid  market.delayTime market.epic market.expiry  market.high   market.instrumentName market.instrumentType  market.lotSize  market.low market.marketStatus  market.netChange  market.offer  market.percentageChange  market.scalingFactor  market.streamingPricesAvailable market.updateTime market.updateTimeUTC  position.contractSize  position.controlledRisk     position.createdDate position.createdDateUTC position.currency position.dealId position.dealReference position.direction  position.level position.limitLevel position.limitedRiskPremium  position.size position.stopLevel position.trailingStep position.trailingStopDistance
    0     1719.59                 0           -             -      1749.95  Sverige30 Cash (100SK)               INDICES           100.0     1713.33           TRADEABLE            -25.44       1721.09                    -1.46                     1                             True          22:15:16             21:15:16                  100.0                    False  2020/07/13 23:12:53:000     2020-07-13T21:12:53               USD                                                       BUY         1720.74                None                        None            5.0               None                  None                          None
    1     1719.59                 0           -             -      1749.95  Sverige30 Cash (100SK)               INDICES           100.0     1713.33           TRADEABLE            -25.44       1721.09                    -1.46                     1                             True          22:15:16             21:15:16                  100.0                    False  2020/07/13 23:12:53:000     2020-07-13T21:12:53               USD                                                       BUY         1720.74                None                        None            5.0               None                  None                          None
    2     1719.59                 0           -             -      1749.95  Sverige30 Cash (100SK)               INDICES           100.0     1713.33           TRADEABLE            -25.44       1721.09                    -1.46                     1                             True          22:15:16             21:15:16                  100.0                    False  2020/07/13 23:12:53:000     2020-07-13T21:12:53               USD                                                       BUY         1720.74                None                        None            5.0               None                  None                          None
    3     1719.59                 0           -             -      1749.95  Sverige30 Cash (100SK)               INDICES           100.0     1713.33           TRADEABLE            -25.44       1721.09                    -1.46                     1                             True          22:15:16             21:15:16                  100.0                    False  2020/07/13 23:12:53:000     2020-07-13T21:12:53               USD                                                       BUY         1720.74                None                        None            5.0               None                  None                          None
    

    data with multiple records

    data =  {
                'positions': [{
                        'market': {
                            'bid': 1719.59,
                            'delayTime': 0,
                            'epic': '-',
                            'expiry': '-',
                            'high': 1749.95,
                            'instrumentName': 'Sverige30 Cash (100SK)',
                            'instrumentType': 'INDICES',
                            'lotSize': 100.0,
                            'low': 1713.33,
                            'marketStatus': 'TRADEABLE',
                            'netChange': -25.44,
                            'offer': 1721.09,
                            'percentageChange': -1.46,
                            'scalingFactor': 1,
                            'streamingPricesAvailable': True,
                            'updateTime': '22:15:16',
                            'updateTimeUTC': '21:15:16'
                        },
                        'position': {
                            'contractSize': 100.0,
                            'controlledRisk': False,
                            'createdDate': '2020/07/13 23:12:53:000',
                            'createdDateUTC': '2020-07-13T21:12:53',
                            'currency': 'USD',
                            'dealId': '',
                            'dealReference': '',
                            'direction': 'BUY',
                            'level': 1720.74,
                            'limitLevel': None,
                            'limitedRiskPremium': None,
                            'size': 5.0,
                            'stopLevel': None,
                            'trailingStep': None,
                            'trailingStopDistance': None
                        }
                    }, {
                        'market': {
                            'bid': 1719.59,
                            'delayTime': 0,
                            'epic': '-',
                            'expiry': '-',
                            'high': 1749.95,
                            'instrumentName': 'Sverige30 Cash (100SK)',
                            'instrumentType': 'INDICES',
                            'lotSize': 100.0,
                            'low': 1713.33,
                            'marketStatus': 'TRADEABLE',
                            'netChange': -25.44,
                            'offer': 1721.09,
                            'percentageChange': -1.46,
                            'scalingFactor': 1,
                            'streamingPricesAvailable': True,
                            'updateTime': '22:15:16',
                            'updateTimeUTC': '21:15:16'
                        },
                        'position': {
                            'contractSize': 100.0,
                            'controlledRisk': False,
                            'createdDate': '2020/07/13 23:12:53:000',
                            'createdDateUTC': '2020-07-13T21:12:53',
                            'currency': 'USD',
                            'dealId': '',
                            'dealReference': '',
                            'direction': 'BUY',
                            'level': 1720.74,
                            'limitLevel': None,
                            'limitedRiskPremium': None,
                            'size': 5.0,
                            'stopLevel': None,
                            'trailingStep': None,
                            'trailingStopDistance': None
                        }
                    }, {
                        'market': {
                            'bid': 1719.59,
                            'delayTime': 0,
                            'epic': '-',
                            'expiry': '-',
                            'high': 1749.95,
                            'instrumentName': 'Sverige30 Cash (100SK)',
                            'instrumentType': 'INDICES',
                            'lotSize': 100.0,
                            'low': 1713.33,
                            'marketStatus': 'TRADEABLE',
                            'netChange': -25.44,
                            'offer': 1721.09,
                            'percentageChange': -1.46,
                            'scalingFactor': 1,
                            'streamingPricesAvailable': True,
                            'updateTime': '22:15:16',
                            'updateTimeUTC': '21:15:16'
                        },
                        'position': {
                            'contractSize': 100.0,
                            'controlledRisk': False,
                            'createdDate': '2020/07/13 23:12:53:000',
                            'createdDateUTC': '2020-07-13T21:12:53',
                            'currency': 'USD',
                            'dealId': '',
                            'dealReference': '',
                            'direction': 'BUY',
                            'level': 1720.74,
                            'limitLevel': None,
                            'limitedRiskPremium': None,
                            'size': 5.0,
                            'stopLevel': None,
                            'trailingStep': None,
                            'trailingStopDistance': None
                        }
                    }, {
                        'market': {
                            'bid': 1719.59,
                            'delayTime': 0,
                            'epic': '-',
                            'expiry': '-',
                            'high': 1749.95,
                            'instrumentName': 'Sverige30 Cash (100SK)',
                            'instrumentType': 'INDICES',
                            'lotSize': 100.0,
                            'low': 1713.33,
                            'marketStatus': 'TRADEABLE',
                            'netChange': -25.44,
                            'offer': 1721.09,
                            'percentageChange': -1.46,
                            'scalingFactor': 1,
                            'streamingPricesAvailable': True,
                            'updateTime': '22:15:16',
                            'updateTimeUTC': '21:15:16'
                        },
                        'position': {
                            'contractSize': 100.0,
                            'controlledRisk': False,
                            'createdDate': '2020/07/13 23:12:53:000',
                            'createdDateUTC': '2020-07-13T21:12:53',
                            'currency': 'USD',
                            'dealId': '',
                            'dealReference': '',
                            'direction': 'BUY',
                            'level': 1720.74,
                            'limitLevel': None,
                            'limitedRiskPremium': None,
                            'size': 5.0,
                            'stopLevel': None,
                            'trailingStep': None,
                            'trailingStopDistance': None
                        }
                    }
                ]
            }