Search code examples
pythonsql-serverodbcpyodbcpymssql

Python with MS SQL - truncated output


I try to connect to MSSQL DB with python from Linux box (Python 2.7, Ubuntu 11.04). Output that I receive is truncated to 500 characters. Please, see script and configs below. How it could be resolved? The problem I suppose in ODBC driver or near it.

The code (pyodbc, pymssql):

conn = pymssql.connect(host='my_remote_host', user='ro_user',
password='password', database='current', as_dict=True)
cur = conn.cursor()
cur.execute(sql)
for i in cur:
    print i
conn.close()

cnxn = pyodbc.connect(driver='FreeTDS', server='my_remote_host', database='current', uid='ro_user', pwd='password')
cursor = cnxn.cursor()
cursor.execute(sql)
rows = cursor.fetchall()
...
cnxn.close()  

I have no write access to MS SQL DB, it's actually remote server that doesn't belong to our system.

SQL:

sql = '''
        SELECT  Req.ID,
        ShReq.Summary AS [Short Name],
        ShReq.ALM_SharedText AS [Text],
        Req.ContainedBy,
        Req.DocumentID
FROM    CurMKS..ALM_Requirement Req
        JOIN CurMKS..ALM_SharedRequirement ShReq ON Req.[References] = ShReq.ID
        WHERE DocumentID = 1111111'''

The problem is with ShReq.ALM_SharedText field. It's truncated to 255 chars, but using conversions like convert(text,ShReq.ALM_SharedText) AS TEXT and CAST(ShReq.ALM_SharedText AS TEXT) I increase truncating to 500 chars. However there are fields with longer text than 500 chars and they are truncated.

ODBC settings:

/etc/odbc.ini:

[MKS]
#Driver=FreeTDS
Driver=/usr/lib/odbc/libtdsodbc.so
Description=Database
Trace=No
Server=my_remote_host
Port=1433
Database=current
UID=ro_user
PWD=password
TDS Version=8.0

/etc/odbcinst.ini:

[FreeTDS]
Description=FreeTDS
Driver=/usr/lib/odbc/libtdsodbc.so
UsageCount=1

/etc/freetds/freetds.conf:

[global]
        tds version = 8.0
;       dump file = /tmp/freetds.log
;       debug flags = 0xffff
;       timeout = 10
;       connect timeout = 10
;       text size = 2097152


[mksserver]
      host = my_remote_host
      port = 1433
      tds version = 8.0
      client charset = UTF-8

Any thoughts how it could be resolved?


Solution

  • Change the text size in global section of freetds.conf to the maximum (4294967295 bytes):

    [global]
        tds version = 8.0
        text size = 4294967295
    

    Also have to set TEXTSIZE in SQL to maximum (2147483647 bytes):

    sql = """
        SET TEXTSIZE 2147483647;
        SELECT  Req.ID,
                ShReq.Summary AS [Short Name],
                ShReq.ALM_SharedText AS [Text],
                Req.ContainedBy,
                Req.DocumentID
        FROM    CurMKS..ALM_Requirement Req
                JOIN CurMKS..ALM_SharedRequirement ShReq ON Req.[References] = ShReq.ID
        WHERE DocumentID = 111111;
          """