Search code examples
pythonmysqlcsvsqlalchemyodo

load CSV into MySQL table with ODO python package - date error 1292


I'm trying to import a simple CSV file that I downloaded from Quandl into a MySQL table with the odo python package

t = odo('C:\ProgramData\MySQL\MySQL Server 5.6\Uploads\WIKI_20160725.partial.csv', 'mysql+pymysql://' + self._sql._user+':'
            + self._sql._password +'@localhost/testDB::QUANDL_DATA_WIKI')

The first row looks like this in the CSV:

A   7/25/2016   46.49   46.52   45.92   46.14   1719772 0   1   46.49   46.52   45.92   46.14   1719772

The MySQL table is defined as follows:

Ticker varchar(255) NOT NULL,
Date date NOT NULL,
Open numeric(15,2) NULL,
High numeric(15,2) NULL,
Low numeric(15,2) NULL,
Close numeric(15,2) NULL,
Volume bigint NULL,
ExDividend numeric(15,2), 
SplitRatio int NULL,
OpenAdj numeric(15,2) NULL,
HighAdj numeric(15,2) NULL,
LowAdj numeric(15,2) NULL,
CloseAdj numeric(15,2) NULL,
VolumeAdj bigint NULL,
PRIMARY KEY(Ticker,Date)

It throws an exception 1292 with the following info:

sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1292, "Incorrect date value: '7/25/2016' for column 'Date' at row 1") [SQL: 'LOAD DATA INFILE %(path)s\n INTO TABLE QUANDL_DATA_WIKI\n CHARACTER SET %(encoding)s\n FIELDS\n TERMINATED BY %(delimiter)s\n ENCLOSED BY %(quotechar)s\n ESCAPED BY %(escapechar)s\n LINES TERMINATED BY %(lineterminator)s\n IGNORE %(skiprows)s LINES\n '] [parameters: {'path': 'C:\ProgramData\MySQL\MySQL Server 5.6\Uploads\WIKI_20160725.partial.csv', 'quotechar': '"', 'skiprows': 0, 'lineterminator': '\r\n', 'escapechar': '\', 'delimiter': ',', 'encoding': 'utf8'}]

Does anyone have an idea what is wrong with the date in the first row? It doesn't seem to match it to the MySql database


Solution

  • mysql has problems with date conversions. I noticed when I defined the date field as a varchar

    Date varchar(255) NOT NULL
    

    then the csv file was read properly

    in my SQL the conversion of the string to date format then looks like this:

    STR_TO_DATE(Date, "%m/%d/%Y")