I have an RDS database where there is a single sql table and new timeseries data shows up in it every 3 hours.
I am trying to make a python script that pulls me all rows of data that came after a certain timestamp (for example t=04/03/2022 21:45:54)?
I tried to look for resources online but I am confused, what Boto3 functions I need to use for this? And what should be my example query?
Here is how I solved the main thing in this question. This code pulls all the rows from the RDS SQL Database that come after a certain timestamp (oldTimestamp). On the first search I found that pyodbc does the job, but it took me sometime to get it to work. One needs to be very careful with string formatting in the pyodbc.connect() function and the string format of sql query. With these 2 things handled well, this should work for you very smoothly. Cheers!
import pyodbc
import pandas as pd
server = 'write your server endpoint in here'
username = 'yourusername'
password = 'yourpassword'
database = 'nameofdatabase'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
oldTimestamp = '2022-04-22 23:30:00'
sql = "SELECT * FROM dbo.eq_min WHERE dbo.eq_min.Timestamp > '" +"{}' ORDER BY dbo.eq_min.Timestamp ASC".format(oldTimestamp)
df = pd.read_sql(sql, cnxn)
print(df.head())