Search code examples
pythonpython-3.xpandasftpftplib

Read FTP file contents in Python and use it at the same time for Pandas and directly

I am trying to download a file from an FTP server in memory, transform it to a dataframe but also return it as bytes. Code as follows:

import io
import pandas as pd
from ftplib import FTP

ftp_connection.cwd(ftp_folder)
download_file = io.BytesIO()
ftp_connection.retrbinary('RETR ' + str(file_name), download_file.write)
download_file.seek(0)
file_to_process = pd.read_csv(download_file, engine='python')

After searching on Stack Overflow, the suggestion was to just read the io stream:

download_file.read()
ValueError: I/O operation on closed file.

Not sure what to try next, without writing the file somewhere and reading it again as bytes.

Solution

  • read_csv probably closes the "file". So read it before you call read_csv:

    download_file.seek(0)
contents = download_file.read()
download_file.seek(0)
file_to_process = pd.read_csv(download_file, engine='python')