Search code examples
amazon-web-servicesamazon-redshiftaws-data-wrangler

wr.redshift.to_sql failed in AWS Data Wrangler2.12.1

awswrangler 2.12.1

I am able to write data.head() into the db, but got error when trying to write all the data. The data is copied from another table and did some cleaning before to_sql. I also did data = data.fillna(value=np.nan)

wr.redshift.to_sql(data, con, schema="level0",
                   table="test", mode="overwrite")

ProgrammingError: {'S': 'ERROR', 'C': '22001', 'M': 'value too long for type character varying(256)', 'F': '/home/ec2-user/padb/src/pg/src/backend/utils/adt/varchar.c', 'L': '511', 'R': 'varchar'}

Solution

  • This is a Redshift error and not linked to AWS Data Wrangler. The exception raised is saying that one of the values you are attempting to write is exceeding the max characters allowed by the column type definition (varchar(256)). More in the docs