Track multiple folders to download files by Spring Integration Sftp

Question

I tried to change the application to track multiple folders and get the files from Sftp.

It is built on the newest Spring stack, Spring Boot 3.2.0, Spring Integration 6.2.0, etc.

Follow the official guide, https://docs.spring.io/spring-integration/reference/sftp/rotating-server-advice.html, when removing remoteDirectory and adding advice settings to the pollor argument sftp inbound adapter,

    @Bean
    fun advice(): RotatingServerAdvice {
        val keyDirectories = listOf(
            RotationPolicy.KeyDirectory(
                "foo",
                sftpProperties.remoteDirectory
            ),
            RotationPolicy.KeyDirectory(
                "foo",
                sftpProperties.remoteFooDirectory
            )
        )
        return RotatingServerAdvice(DelegatingSessionFactory { sftpSessionFactory() }, keyDirectories, true)
    }


    @Bean
    fun sftpInboundFlow(): IntegrationFlow {
        return IntegrationFlow
            .from(
                Sftp.inboundAdapter(sftpSessionFactory())
                    .preserveTimestamp(true)
                    .deleteRemoteFiles(true) // delete files after transfer is done successfully
                    // use advice to select multiple folders.
                    //.remoteDirectory(sftpProperties.remoteDirectory) // removed this line.
                    .regexFilter(".*\\.csv$")
                    // local settings
                    .localFilenameExpression("#this.toUpperCase() + '.csv'")
                    .autoCreateLocalDirectory(true)
                    .localDirectory(File("./sftp-inbound"))
            ) { e: SourcePollingChannelAdapterSpec ->
                e.id("sftpInboundAdapter")
                    .autoStartup(true)
                    .poller(Pollers.fixedDelay(5000).advice(advice()))
            }

It failed due to exception like this running the tests, https://github.com/hantsy/spring-puzzles/blob/master/integration-sftp/src/test/kotlin/com/example/demo/SftpIntegrationFlowsTestWithEmbeddedSftpServer.kt#L122l.

 'remoteDirectoryExpression' must not be null
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:608)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:523)
    at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:946)

The complete project source codes, https://github.com/hantsy/spring-puzzles/tree/master/integration-sftp

Answer 1

Looks like you have missed that config from the example in the docs:

.remoteDirectory(".")

Even if we use that RotatingServerAdvice, we still have to provide that required option. The channel adapter does not know about advice. Although that remoteDirectory(".") is going to be overridden on each poll according to the advice config.