Search code examples
spring-integrationspring-integration-dslspring-integration-sftp

Filter and get list of remote file names from Sftp


I need to get list of remote file names from sftp server whose modified time is more than some given time and then delete all those files from sftp, I'm using SftpOutboundGateway's NLST command to list the file names

But it gets all files from the given remote dir path, I tried .filter(lastModifiedFileListFilter) that didn't work then I also tried .filterFunction(logicGiven) didn't work either

I wonder if NLST command support filtering

Here is the code sample:

Filter instance:

private FileListFilter lastModifiedFilter(){
    LastModifiedFileListFilter fileListFilter = new LastModifiedFileListFilter();
    fileListFilter.setAge(120, TimeUnit.SECONDS);
    return fileListFilter;
}

With filter() method:

  @Bean
public IntegrationFlow deleteFiles(){
    return IntegrationFlows.from("integration.channel.bulk-delete")
            .handle(Sftp.outboundGateway(sftpSessionFactory(),
            AbstractRemoteFileOutboundGateway.Command.NLST,"headers[path]")
                    .filter(lastModifiedFilter()))
            .get();

with filterFunction() method:

 @Bean
public IntegrationFlow deleteFiles(){
    return IntegrationFlows.from("integration.channel.bulk-delete")
            .handle(Sftp.outboundGateway(sftpSessionFactory(),
            AbstractRemoteFileOutboundGateway.Command.NLST,"headers[path]")
                    .filterFunction(file->{
                        Instant decidedTime = Instant.now().minus(120, ChronoUnit.SECONDS);
                       return Instant.ofEpochSecond(file.getAttrs().getMTime()).isBefore(decidedTime);
                    }))
            .log()
            .get();

How can I filter and get file names. I dont want to use mget command

UPDATE

The RM command in the next handle method doesn't delete, I also placed a break point in doRm() method it doesn't stop there though I have files in my for loop

@Bean
public IntegrationFlow deleteFiles(){
    return IntegrationFlows.from("integration.channel.bulk-delete")
            .handle(Sftp.outboundGateway(sftpSessionFactory(),
            AbstractRemoteFileOutboundGateway.Command.LS,"headers[path]")
                    .options(AbstractRemoteFileOutboundGateway.Option.NAME_ONLY)
                    .filterFunction(file->{
                        Instant decidedTime = Instant.now().minus(120, ChronoUnit.SECONDS);
                        return Instant.ofEpochSecond(file.getAttrs().getMTime()).isBefore(decidedTime);
                    }))
            .log()
            .handle(fileList->{
                List<String> files = (List<String>) fileList.getPayload();
                for (String remoteFile: files) {
                    Sftp.outboundGateway(sftpSessionFactory(),
                            AbstractRemoteFileOutboundGateway.Command.RM,
                            "headers[file_remoteDirectory]+"+remoteFile);
                }
            })
            .get();

Solution

  • NLST only fetches the file names so there is no timestamp to filter on.

    NLST currently applies no filters at all.

    Use LS (possibly with the recursive -R option).