Search code examples
pythonsftpparamiko

How to download all the files from today from SFTP server with Paramiko?


I am beginner in Python and I am trying to download all files, that were added today from SFTP server using Paramiko. I have found this question which explains how to download the latest file:
How to download only the latest file from SFTP server with Paramiko?
However, I was not able to get it running and I need to download all files that were added today, not the latest one.

I am able to connect and download the files one by one giving the exact path but I would like to automate the process. The code I tried is a combination of what I have found on the internet:

import paramiko
import functools

paramiko.util.log_to_file("paramiko.log")

# Open a transport
host,port = "sftp.xxxxx.com",22
transport = paramiko.Transport((host,port))

# Auth
username,password = "yyyyy","bbbbbb"
transport.connect(None,username,password)

# Go!
sftp = paramiko.SFTPClient.from_transport(transport)

# Download
filepath = "/out/filenames.csv"
localpath = "C:/data_transfered/"
sftp.get(filepath,localpath)

latest = 0
latestfile = None

for fileattr in sftp.listdir_attr():
    if fileattr.filename.startswith('JOY') and fileattr.st_mtime \> latest:
        latest = fileattr.st_mtime
        latestfile = fileattr.filename

if latestfile is not None:
    sftp.get(latestfile, latestfile)

I don't know where and how exactly to specify the source filepath and folder for downloaded files (localpath). Also need to download all files from today. Then I will execute shell commands the code everyday in order to get all files on daily basis.


Solution

  • Calculate today's midnight and compare it against remote file timestamp (file.st_mtime):

    localpath = "C:/data_transfered"
    remotepath = "/out"
    
    today_midnight = datetime.combine(datetime.today(), time.min)
    
    for fileattr in sftp.listdir_attr(remotepath):
        mtime = datetime.fromtimestamp(fileattr.st_mtime)
        if fileattr.filename.startswith('JOY') and (mtime >= today_midnight):
            remotefilepath = remotepath + "/" + fileattr.filename
            localfilepath = os.path.join(localpath, fileattr.filename)
            sftp.get(remotefilepath, localfilepath)
    

    Similar question: How do I download files from yesterday with Paramiko?