Is there a way I send mail with all files (as Zip) received in Spring Integration FTP instead sending the files Individually per Message received

Question

I have a Spring Boot application based on Spring Integration FTP where I can DOWNLOAD/UPLOAD files.

I need to send the files (as ZIP) received via mail but the problem is, I can only do this per message a file received which means if I receive 100 files, the email recipient would receive 100 emails. This is because Spring Integration processes the messages one after the other.

I need a way to aggregate the files received per poll and send to the recipient instead of sending the files one after the other.

I have tried to come up with some ideas such as these:

1. move the files received to a temporary location as they are received, check when the poll is done, zip the files in this temporary location, and send them to the recipient.
2. Find out how to know files received per poll.

Please can someone advise on how I can implement this:

public IntegrationFlow createIntFlow(MessageDirectory directory, ConcurrentMetadataStore metadataStore) {
        var directoryName = directory.getDirectoryName();

        var integrationFlowBuilder = IntegrationFlows
                .from(getInboundAdapter(directory, metadataStore),
                        e -> e.id(directoryName + "-PerPoller")
                                .autoStartup(true)
                                .poller(Pollers
                                        .cron(directory.getSchedule())
                                        .taskExecutor(simpleAsyncTaskExecutor)
                                        .errorChannel("errorChannel")
                                        .maxMessagesPerPoll(-1)))
                .log()
         

        **integrationFlowBuilder.publishSubscribeChannel(s -> {
           s.subscribe(sf -> sf.handle(c -> emailEmitter.createFileEvent(directory, c.getPayload().toString())));**
       });

            });
        }
        return integrationFlowBuilder.get();
    }```

Answer 1

See this project https://github.com/spring-projects/spring-integration-extensions/tree/main/spring-integration-zip. There is a ZipTransformer which can zip an Iterable input message.

So, probably a source polling channel adapter does not fit to your requirements, but better to look into an FtpOutboundGateway and its MGET command support: https://docs.spring.io/spring-integration/docs/current/reference/html/ftp.html#using-the-mget-command. So, you got a List<File> as a result of this call and you send this message to that ZipTransformer. The result if zip is just one zip file which you simply can send to email.

You still may use a source polling channel adapter, but rather as a plain trigger via simple Supplier - () -> "". Or with a remote dir as a payload for that FtpOutboundGateway.

You may also consider to use a AcceptOnceFileListFilter as a transformer step before zipping to filter out those files you have processed before.