Alpha Vantage stockinfo only collects 4 dfs properly formatted, not 6

Question

I can get 4 tickers of stockinfo from Alpha Vantage before the rest of the DataFrames are not getting the stockinfo I ask for. So my resulting concatenated df gets interpreted as Nonetype (because the 4 first dfs are formatted differently than the last 2). This is not my problem. The fact that I only get 4 of my requests is... If I can fix that - the resulting concatenated df will be intact.

My code

import pandas as pd
import datetime
import requests
from alpha_vantage.timeseries import TimeSeries
import time

tickers = []

def alvan_csv(stocklist):
    api_key = 'demo'   # For use with Alpha Vantage stock-info retrieval.
    for ticker in stocklist:
        #data=requests.get('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=%s&apikey={}'.format(api_key) %(ticker))
        df = pd.read_csv('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&datatype=csv&symbol=%s&apikey={}'.format(api_key) %(ticker))#, index_col = 0)  &outputsize=full
        df['ticker'] = ticker
        tickers.append(df)
        # concatenate all the dfs
        df = pd.concat(tickers)
        print('\ndata before json parsing for === %s ===\n%s' %(ticker,df))
        df['adj_close'] = df['adjusted_close']
        del df['adjusted_close']
        df['date'] = df['timestamp']
        del df['timestamp']
        df = df[['date','ticker','adj_close','volume','dividend_amount','split_coefficient','open','high','low']] #
        df=df.sort_values(['ticker','date'], inplace=True)
        time.sleep(20.3)
    print('\ndata after col reshaping for === %s ===\n%s' %(ticker,df))
    return df


if __name__ == '__main__':
    stocklist = ['vws.co','nflx','mmm','abt','msft','aapl']

    df = alvan_csv(stocklist)

NB. Please note that to use the Alpha Vantage API, you need a free API-Key which you may optain here: https://www.alphavantage.co/support/#api-key Replace the demo API Key with your API Key to make this code work.

Any ideas as to get this to work?

Answer 1

Apparently Alpha Vantage has a pretty low fair usage allowance, where they measure no of queries pr. minute. So in effekt only the first 4 stocks are allowed at full speed. The rest of the stocks need to pause before downloading for not violating their fair-usage policy.

I have now introduced a pause between my stock-queries. At the moment I get approx 55% of my stocks, if I pause for 10 sec. between calls, and 100% if I pause for 15 seconds.

I will be testing exactly how low the pause can be set to allow for 100% of stocks to come through.

I must say compared to the super high-speed train we had at finance.yahoo.com, this strikes me as steam-train. Really really slow downloads. To get my 500 worth of tickers it takes me 2½ hours. But I guess beggars can't be choosers. This is a free service and I will manage with this.