I am using the code below to pull data from Google Finance. The timestamp is in POSIX form, so it is converted into data time. When I try to filter it based on a time criteria (14:35:00), it returns an empty table. I suspect it has to do with the POSIX/ datetime conversion, but have no idea how to resolve it.
def get_intraday_data(symbol, interval_seconds=301, num_days=10):
# Specify URL string based on function inputs.
url_string = 'http://www.google.com/finance/getprices?q={0}'.format(symbol.upper())
url_string += "&i={0}&p={1}d&f=d,o,h,l,c,v".format(interval_seconds,num_days)
# Request the text, and split by each line
r = requests.get(url_string).text.split()
# Split each line by a comma, starting at the 8th line
r = [line.split(',') for line in r[7:]]
# Save data in Pandas DataFrame
df = pd.DataFrame(r, columns=['Datetime','Close','High','Low','Open','Volume'])
# Convert UNIX to Datetime format
df['Datetime'] = df['Datetime'].apply(lambda x: datetime.datetime.fromtimestamp(int(x[1:])))
#Seperate Date and Time
df['Time'],df['Date']= df['Datetime'].apply(lambda x:x.time()), df['Datetime'].apply(lambda x:x.date())
#Convert 'Close','High','Low','Open', deleting 'Volume'
''''df['Close'] = df['Close'].astype('float64')
df['High'] = df['High'].astype('float64')
df['Low'] = df['Low'].astype('float64')
df['Open'] = df['Open'].astype('float64')'''
del df['Volume']
del df['Datetime']
df[['Close','High','Low','Open']] = df[['Close','High','Low','Open']].astype('float64')
# Calculating %Change and Range
df['%pct'] = (df['Close'] - df['Open'])/df['Open']
df['Range'] = df['High'] - df['Low']
#Sort Columns
return df
I have stored the results of this function as NAS
NAS = get_intraday_data('IXIC', interval_seconds=301, num_days= 100)
The Filter Criteria is:
NAS[NAS['Time'] == '14:35:00']
I will appreciate assistance on this.
I see that you are converting timestamp
to datetime
incorrectly. You are calling datetime
twice.
Replace
df['Datetime'] = df['Datetime'].apply(lambda x: datetime.datetime.fromtimestamp(int(x[1:])))
with
df['Datetime'] = df['Datetime'].apply(lambda x: datetime.fromtimestamp(int(x[1:])))
In 2nd part of your question:
NAS = get_intraday_data('IXIC', interval_seconds=301, num_days= 100)
NAS[NAS['Time'] == '14:35:00']
You care comparing instance of datetime.time
with string, which is not correct. Try
NAS[NAS['Time'] == datetime.strptime('14:35:00', '%H:%M:%S').time()]
it should work as expected.
Update:
Running script with suggested changes will display data as:
Close High Low Open Time Date %pct
60 5162.448 5165.124 5162.448 5165.057 14:35:00 2016-07-29 -0.000505
138 5181.768 5183.184 5181.193 5181.404 14:35:00 2016-08-01 0.000070
216 5130.514 5131.933 5130.434 5131.893 14:35:00 2016-08-02 -0.000269
294 5146.608 5146.608 5143.827 5144.788 14:35:00 2016-08-03 0.000354
372 5163.854 5164.154 5162.997 5164.021 14:35:00 2016-08-04 -0.000032
450 5221.624 5221.911 5220.658 5220.789 14:35:00 2016-08-05 0.000160
528 5204.111 5204.240 5202.476 5202.865 14:35:00 2016-08-08 0.000239
.
.
.
3648 5282.999 5283.017 5279.008 5279.340 14:35:00 2016-10-04 0.000693
3726 5324.450 5325.375 5323.628 5324.129 14:35:00 2016-10-05 0.000060
3804 5310.945 5311.454 5310.194 5310.558 14:35:00 2016-10-06 0.000073
3882 5295.064 5295.080 5292.184 5292.327 14:35:00 2016-10-07 0.000517