Convert paramiko.sftp_file.SFTPFile object to unicode

Question

What I need, is to copy a .csv file from an SFTP server to a Google Cloud Storage bucket without downloading the file locally.

Here is my code so far:

from google.cloud import storage 
import io 
import paramiko

# connection parameters
host = 'host'
port = 22
usr = 'me'
pwd = '0123'

# Connect to the SFTP server
sftp_client = paramiko.SSHClient()
sftp_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
sftp_client.connect(host, port, usr, pwd)
sftp = sftp_client.open_sftp()

# Get the file
sftp_file =  sftp.open('./file.csv', 'rb')

# Call the Google Cloud Storage API
storage_client = storage.Client()
bucket_name = 'the_bucket'
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(sftp_file)

# Upload to bucket
blob.upload_from_filename(sftp_file)

And here is the error returned:

ValueError: <paramiko.sftp_file.SFTPFile object at 0x0000022665279A00> could not be converted to unicode

Can someone unstuck me?

Answer 1

Use Blob.upload_from_file to upload the file-like object that you have opened using SFTPClient.open:

with sftp.open('./file.csv', 'rb') as sftp_file:
    sftp_file.prefetch()
    blob.upload_from_file(sftp_file)

---

For the purpose of SFTPFile.prefetch, see Reading file opened with Python Paramiko SFTPClient.open method is slow.

---

And just to be very clear, this code still downloads the file from SFTP server to the local machine – it just does not store it to a local file.