Search code examples
springspring-integrationfilteringspring-integration-sftp

Spring Integration File synchronizer : Accept files based on a pre defined list


I am transfering files from a remote to a local by Sftp, for processing. I want to only transfer .csv files, and I have a list of pre-defined filenames.

I couldn't find a FileListFilter that allows to specify multiple patterns and transfer if at least one if matched.

So far I have this code, that's woorking for ".csv" filtering.

The Integration Flow

@Bean
    public IntegrationFlow integFlow() {
        return IntegrationFlows
            .from(ftpMessageSource(), c -> poller())
            ... more processing

The MessageSource

public MessageSource<File> ftpMessageSource() {

            SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sessionFactory);
            fileSynchronizer.setRemoteDirectory(remoteDirectory);
            fileSynchronizer.setDeleteRemoteFiles(true);
            fileSynchronizer.setFilter(new SftpRegexPatternFileListFilter(Constantes.EXTENSION));
            SftpInboundFileSynchronizingMessageSource ftpInboundFileSync = 
                new SftpInboundFileSynchronizingMessageSource(fileSynchronizer);

            ftpInboundFileSync.setLocalDirectory(new File(workDirectory));
            ftpInboundFileSync.setAutoCreateLocalDirectory(true);
            CompositeFileListFilter<File> compositeFileListFilter = new CompositeFileListFilter<>();
            compositeFileListFilter.addFilter(new RegexPatternFileListFilter(Constantes.EXTENSION));
            ftpInboundFileSync.setLocalFilter(compositeFileListFilter);
            return ftpInboundFileSync;
    }

Constantes.EXTENSION is a regex accepting .csv and .CSV. This works fine.

Say that I have a String list that contains "string1',"string2","string3" and I want to transfer every file of the form string1*, string2* or string3*. How would I proceed ?


Solution

  • @SpringBootApplication
    public class So59161698Application {
    
        public static void main(String[] args) {
            SpringApplication.run(So59161698Application.class, args);
        }
    
        private final String myPatterns = "foo,bar,baz";
    
        @Bean
        public FileListFilter<File> filter() {
            Set<String> patterns = StringUtils.commaDelimitedListToSet(this.myPatterns);
            return files -> Arrays.stream(files)
                    .filter(file -> patterns.stream()
                            .filter(pattern -> file.getName().startsWith(pattern))
                            .findFirst()
                            .isPresent())
                    .collect(Collectors.toList());
        }
    
        @Bean
        public ApplicationRunner runner(FileListFilter<File> filter) {
            return args -> {
                System.out.println(filter.filterFiles(new File[] {
                        new File("foo.csv"),
                        new File("bar.csv"),
                        new File("baz.csv"),
                        new File("qux.csv")
                }));
            };
        }
    
    }
    
    [foo.csv, bar.csv, baz.csv]