Search code examples
pythoncsvftplib

new line chars added to csv file after ftp.storbinary()


I am attempting to store a csv file on an ftp server using python's ftplib module. Right now, I have about 30 lines of code which generates probabilities of weather values in a 2-d array. I then write this 2-d array to a csv file.

When I write the csv file onto my local drive, the file displays as expected within excel. However, when I view the file after I uploaded it to an ftp server, I see that a new line character has been added after every row.

I've done some minor testing to see what the problem may be, and I have been able to upload the csv file with coreftp. The csv file displays correctly after I do that. So I am pretty sure the file is fine, its something that is happening when python uploads it onto an ftp server.

I was originally creating a text file with a .csv extension file then reopening it as a binary file and uploading it. I thought that may be the issue so I tried using the csv module, but same issue.

Here is my code at the moment...

TEMPSHEADER = [i-50 for i in range(181)]#upper bounds exclusive
WINDSHEADER = [i for i in range(101)]#upper bounds exclusive
HEADER = TEMPSHEADER + WINDSHEADER
for site in ensmosdic:
    ensmos = ensmosdic.get(site)
    with open(utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.csv","w",newline='') as csvfile:
        writer = csv.writer(csvfile, delimiter=",")
        writer.writerow(["CODE        ","F","ForecastDate","HOUR"]+HEADER)
        siteTable =[[0 for x in range(286)] for y in range(24,169)]#upper bounds exclusive
        ###########
        #other code here, but not important with regards to post
        ###########
        for i in siteTable:
            writer.writerow(i)

        csvfile.close()#not sure if you have to close csv file, not in csv module docs
    f = open(utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.csv","rb")
    ftpInno.storbinary("STOR " + utcnow.strftime("%Y-%m-%d-") + site +"-prob.csv",f)
    f.close()
ftpInno.close()

Thanks in advance


Solution

  • After an hour or so of trouble shooting, the answer is fairly simple, although I am not entirely sure why it works.

    what i did was create a text file instead of a csv file which I was doing in my original question

    with open(FILELOCATION + utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.txt","w") as f:
        #write to the file below
        f.close()
    #open file again as a txt file
    f = open(FILELOCATION + utcnow.strftime("%Y-%m-%d") + "-" +site+"-prob.txt","rb")
    ftp.storlines("STOR " + utcnow.strftime("%Y-%m-%d-") + site +"-prob.csv",f)
    f.close()
    

    reading the file as a binary file and then storing it with the storlines method removed the extra lines I was seeing within the file after I uploaded it to an ftp server.