Spring batch running in a docker container to Write file out of the docker container - directly to the SFTP remote server

Question

My spring batch job running in a docker container reads data from DB and creates a flatfile. For now the file is created inside the container but when data is big, i want to create the flatfile in the remote SFTP location. What would be the best way to implement this without creating a physical file inside a container.

Answer 1

I would use the org.apache.commons.net.ftp.FTPClient class, storeFileStream method to obtain an OutputStream, then write directly to that OutputStream when reading rows from your database.

See https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html#storeFileStream-java.lang.String-