Search code examples
pythonsql-serverpandasstored-procedurespandasql

Pandas IO SQL and stored procedure with multiple result sets

So I have a stored proc on a local sql server, this returns multiple data sets / tables

Normally, in python / pyodbc I would use 

cursor.nextset()
subset1 = cursor.fetchall()
cursor.nextset()
subset2 = cursor.fetchall()

I wish to make use of the ps.io.sql.read_sql and return the stored procedure with multiple result sets into dataframes, however I can not find anything that references how to move the cursor along and get more information before closing things off.

import pandas as ps

query = "execute raw.GetDetails @someParam = '118'"
conn = myConnection() #connection,cursor

results = ps.io.sql.read_sql(query, con=conn[0])

results.head()

conn[1].close()
Solution

  • The following should work:

    import pandas as pd
from sqlalchemy import create_engine

engine = create_engine('mysql://{}:{}@{}/{}'.format(username, password, server, database_name))
connection = engine.connect().connection
cursor = self.connection.cursor()

cursor.execute('call storedProcName(%s, %s, ...)', params)

# Results set 1
column_names = [col[0] for col in cursor.description] # Get column names from MySQL

df1_data = []
for row in cursor.fetchall():
    df1_data.append({name: row[i] for i, name in enumerate(column_names)})

# Results set 2
cursor.nextset()
column_names = [col[0] for col in cursor.description] # Get column names from MySQL

df2_data = []
for row in cursor.fetchall():
    df2_data.append({name: row[j] for j, name in enumerate(column_names)})

cursor.close()

df1 = pd.DataFrame(df1_data)
df2 = pd.DataFrame(df2_data)

    Edit: I've updated the code here to avoid having to manually specify the column names.

    Note that the original question only specifies a "local SQL server", not a specific kind of SQL server. This answer works with MySQL, but I haven't tested it with any other variety.