Search code examples
pythonsql-serverpandassqlalchemypyodbc

Inserting rows into Microsoft SQL Server using pandas raises precision error

I am trying to insert data into a mssql database. I needed as fast method for this so I set the fast_executemany param to true. The upload works fine for most part but if one of the column is a datetime with timezone it crashes raising: (pyodbc.Error) ('HY104', '[HY104] [Microsoft][ODBC Driver 17 for SQL Server]Invalid precision value (0) (SQLBindParameter)') If I insert the same data with fast_executemany as False then everythhing works perfectly. Did anyone came across a similar problem or know what might be the issue?

Sample code

from sqlalchemy import create_engine, engine
import pytz
import pandas as pd
from sqlalchemy.dialects.mssql import DATETIMEOFFSET
import datetime


engine_url = engine.URL.create(
            drivername='mssql',
            username='admin',
            password='**',
            host='**',
            port='1433',
            database='mytestdb1',
            query={'driver': "ODBC Driver 17 for SQL Server"}
        )
mssql_engine = create_engine(engine_url, echo=False, fast_executemany=True)



base = datetime.datetime.today().replace(tzinfo=pytz.utc)
date_list = [base - datetime.timedelta(days=x) for x in range(20)]
df = pd.DataFrame(date_list, columns = ['date_time'])
df.to_sql('test_insert', mssql_engine, schema='testschema1', if_exists='replace', dtype = {'date_time':DATETIMEOFFSET})

response:

DBAPIError: (pyodbc.Error) ('HY104', '[HY104] [Microsoft][ODBC Driver 17 for SQL Server]Invalid precision value (0) (SQLBindParameter)')
[SQL: INSERT INTO testschema1.test_datetime ([index], date_time) VALUES (?, ?)]
[parameters: ((0, '2022-05-06 16:40:05.434984 +00:00'), (1, '2022-05-05 16:40:05.434984 +00:00'), (2, '2022-05-04 16:40:05.434984 +00:00'), (3, '2022-05-03 16:40:05.434984 +00:00'), (4, '2022-05-02 16:40:05.434984 +00:00'), (5, '2022-05-01 16:40:05.434984 +00:00'), (6, '2022-04-30 16:40:05.434984 +00:00'), (7, '2022-04-29 16:40:05.434984 +00:00')  ... displaying 10 of 20 total bound parameter sets ...  (18, '2022-04-18 16:40:05.434984 +00:00'), (19, '2022-04-17 16:40:05.434984 +00:00'))]
 (Background on this error at: http://sqlalche.me/e/14/dbapi)

sqlalchemy==1.4, pyodbc==4.0.32 and pandas==1.2.0

As I said the code works perfectly if I dont use fast_executemany.

Solution

  • This issue can be reproduced using SQLAlchemy 1.4.0. It was fixed in SQLAlchemy 1.4.1.