Spring Integration SFTP (xml) Outbound Gateway - copy folderstructure from remote directory to local directory using SPEL Expression and java

Question

I am totally new to Spring Integration and a beginner in Spring. It is a miracle that I got so far but I already can copy my files recursively into a local folder. I based my app on this sample application from Spring:

https://github.com/spring-projects/spring-integration-samples/blob/main/basic/sftp/src/test/resources/META-INF/spring/integration/SftpOutboundGatewaySample-context.xml

Now I want to keep the same subdirectories as in the remote folder.

I am using Spring 4.1.9 Release, because of legacy code. These are my dependencies.

<dependencies>
    <dependency>
        <groupId>org.springframework.integration</groupId>
        <artifactId>spring-integration-core</artifactId>
        <version>4.1.9.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.integration</groupId>
        <artifactId>spring-integration-sftp</artifactId>
        <version>4.1.9.RELEASE</version>
    </dependency>
</dependencies>

This is a simple App class with main method which loads the applicationcontext.

@Component
public class App {

    public static void main(String[] args) {
        ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext(
                "classpath:SftpOutboundGateway-context-test.xml");
        ToSftpFlowGateway toFtpFlow = ctx.getBean(ToSftpFlowGateway.class);

        toFtpFlow.lsGetAndRmFiles("/remote_directory/");
        
    }
}

This is the interface, and the bean is made in the xml config underneath

public interface ToSftpFlowGateway
{
    List<Boolean> lsGetAndRmFiles(String dir);
}

This is the xml spring config. It has a reference to web.xml where the sessionfactory is made. Here I use the outbound-gateway

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:int="http://www.springframework.org/schema/integration"
       xmlns:int-sftp="http://www.springframework.org/schema/integration/sftp"
       xsi:schemaLocation="http://www.springframework.org/schema/integration
                           http://www.springframework.org/schema/integration/spring-integration-4.1.xsd
                           http://www.springframework.org/schema/beans
                           https://www.springframework.org/schema/beans/spring-beans.xsd
                           http://www.springframework.org/schema/integration/sftp 
                           https://www.springframework.org/schema/integration/sftp/spring-integration-sftp-4.1.xsd">

    <import resource="web.xml"/>
    

    <int:gateway id="gw" 
                 service-interface="ToSftpFlowGateway"
                 default-request-channel="receiveChannel"
    />

    <int-sftp:outbound-gateway id="gateway1" 
                              auto-startup="true"
                              session-factory="sftpSessionFactory"
                              request-channel="receiveChannel"
                              command="mget"
                              local-directory-expression="'C:/temp/' + headers['file_remoteDirectory']"
                              auto-create-local-directory="true"
                              rename-expression="" 
                              command-options="-R" 
                              mode="IGNORE"
                              expression="payload"
                              reply-channel="loggingChannel" 
    >
        <int:poller fixed-rate="1"/>
    </int-sftp:outbound-gateway>
    
    <int:channel id="receiveChannel">
        <int:queue/>
    </int:channel>

    <int:logging-channel-adapter id="loggingChannel" log-full-message="true" logger-name="tapInbound"
                                 level="INFO" />
    
</beans>

I want to know why my subdirectories are null. I want my subdirectories to be C:/temp/in and C:/temp/out as the following picture.

Now i get the following output as logging:

INFO: GenericMessage [payload=[C:\temp\null\file.txt, C:\temp\null\file2.txt, C:\temp\null\file3.txt], headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel@534a53da, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel@534a53da, id=f012840c-184e-203e-1a90-754beb8796d1, file_remoteDirectory=/remote_directory/, file_remoteFile=, timestamp=1684943419215}]

This is probably wrong

local-directory-expression="'C:/temp/' + headers['file_remoteDirectory']"

Spring documentation says the following, but I don't understand:

Typically, you would use the #remoteDirectory variable in the local-directory-expression so that the remote directory structure is retained locally.

Answer 1

The headers['file_remoteDirectory'] expression references to the request message which is definitely doesn't have that info because it is a plain message:

toFtpFlow.lsGetAndRmFiles("/remote_directory/");

Try to modify your local-directory-expression to this:

local-directory-expression="'C:/temp/' + #remoteDirectory"

Where that remoteDirectory SpEL variable is populated for the path of the remote sub-dir against every file retrieved:

private File generateLocalDirectory(Message<?> message, String remoteDirectory) {
    EvaluationContext evaluationContext = ExpressionUtils.createStandardEvaluationContext(getBeanFactory());
    if (remoteDirectory != null) {
        evaluationContext.setVariable("remoteDirectory", remoteDirectory);
    }
    File localDir = ExpressionUtils.expressionToFile(this.localDirectoryExpression, evaluationContext, message,
            "Local Directory");
    if (!localDir.exists()) {
        Assert.isTrue(localDir.mkdirs(), () -> "Failed to make local directory: " + localDir);
    }
    return localDir;
}