Search code examples
pythonmysqlstock-data

Unable to load data into mysql using python


I was not able to load symbol column into mysql. File name is BAJFINANCE.NS.csv

Date,Open,High,Low,Close,Adj Close,Volume   
2002-07-01,5.239740,5.540930,5.239740,5.412680,0.063393,21923  
2002-07-02,5.523440,5.537040,5.421420,5.443770,0.063757,61045  
2002-07-03,5.505950,5.545790,5.328150,5.466120,0.064019,34161  

in mysql am getting

timestamp,open,high,low,close,adj_close,volume,symbol  
2002-07-01,5.23974,5.54093,5.23974,5.41268,0.063393,21923,NULL  
2002-07-02,5.52344,5.53704,5.42142,5.44377,0.063757,61045,NULL  
2002-07-03,5.50595,5.54579,5.32815,5.46612,0.064019,34161,NULL

How to include symbol in mysql

import mysql.connector
config = {
'user':'root',
'password':'root',
'host':'127.0.0.1',
'database':'masterfiles'
 }
 conn= mysql.connector.connect(**config)
 c=conn.cursor()

def insertRows(fileName,c):
    delimiter=r','
    file=fileName.split("/")[-1]
    symbol = file[:-7]
    if file.startswith("BAJ"):
        c.execute("""Load data local infile %s into table adjprice fields terminated by %s ignore 1 lines 
                 (timestamp,open,high,low,close,adj_close,volume,@symbol)""",(fileName,delimiter))

localExtractFilePath="/Users/dalal"

import os

for file in os.listdir(localExtractFilePath):
    if file.endswith("csv"):
        insertRows(localExtractFilePath+"/"+file,c)
        print "Loaded file "+file+" into database"
        conn.commit()
c.close()
conn.close()

Solution

  • I think this will do what you want:

           c.execute("""
    Load data local infile %s
        into table adjprice
        fields terminated by %s
        ignore 1 lines 
        (timestamp, open, high, low, close, adj_close, volume)
        set symbol = %s
    """, (fileName, delimiter, symbol))
    

    I would recommend using query parameters rather than munging the string. However, I'm not 100% sure if the filename and fields components can be passed in as parameters.