I have tried to use emberstack/sftp or atmoz/sftp to host a sftp server in Docker container for testing purpose.
sftp:
image: emberstack/sftp
volumes:
- ./data/sftp.json:/app/config/sftp.json:ro
- ./data/sftptest:/home/demo/sftp:rw
ports:
- "2222:22"
And my project is based on Spring Boot 2.7.4/Kotlin Coroutines/Java17.
I have already defined the following beans for uploading upload the files.
@Confgiuration
class MyConfig{
//other beans.
@Bean
fun sftpOutboundFlow(): IntegrationFlow {
return IntegrationFlows
.from("toSftpChannel")
.handle(
Sftp.outboundAdapter(sftpSessionFactory(), FileExistsMode.FAIL)
.useTemporaryFileName(false)
.remoteDirectory(sftpProperties.remoteDirectory)
)
.get()
}
}
// a messaging gateway to send file.
@MessagingGateway
interface UploadGateway {
@Gateway(requestChannel = "toSftpChannel")
fun upload(file: File)
}
The unit test is like this.
@Test
fun `upload ach batch files to sftp`() = runTest {
val test = File("src/test/resources/foo.txt")
log.debug("uploading file: $test, ${test.exists()}")
uploadGateway.upload(test)
eventually(5.seconds) {
Paths.get("./data/sftptest/foo.txt").shouldExist()
}
}
The above log debug output shows the uploading file is existed as expected.
I got the following exceptions.
Error handling message for file [D:\myproject\build\resources\test\foo.txt -> foo.txt];
nested exception is org.springframework.messaging.MessagingException:
Failed to write to '/home/demo/sftp/foo.txt' while uploading the file; nested exception is java.io.IOException:
failed to write file
org.springframework.messaging.MessageDeliveryException:
Error handling message for file [D:\myproject\build\resources\test\foo.txt -> foo.txt];
nested exception is org.springframework.messaging.MessagingException:
Failed to write to '/home/demo/sftp/foo.txt' while uploading the file;
nested exception is java.io.IOException:
failed to write file, failedMessage=GenericMessage [payload=D:\myproject\build\resources\test\foo.txt,
....
Caused by: org.springframework.messaging.MessagingException: Failed to write to '/home/demo/sftp/foo.txt' while uploading the file; nested exception is java.io.IOException: failed to write file
at app//org.springframework.integration.file.remote.RemoteFileTemplate.sendFileToRemoteDirectory(RemoteFileTemplate.java:573)
at app//org.springframework.integration.file.remote.RemoteFileTemplate.doSend(RemoteFileTemplate.java:353)
... 143 more
Caused by: java.io.IOException: failed to write file
at org.springframework.integration.sftp.session.SftpSession.write(SftpSession.java:176)
at org.springframework.integration.file.remote.session.CachingSessionFactory$CachedSession.write(CachingSessionFactory.java:237)
at org.springframework.integration.file.remote.RemoteFileTemplate.doSend(RemoteFileTemplate.java:601)
at org.springframework.integration.file.remote.RemoteFileTemplate.sendFileToRemoteDirectory(RemoteFileTemplate.java:570)
... 144 more
Caused by: 2: No such file
at app//com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2873)
at app//com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:594)
at app//com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:540)
at app//com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:492)
at app//org.springframework.integration.sftp.session.SftpSession.write(SftpSession.java:173)
... 147 more
Update: I created a sample project to produce this issue.
Run the following command to start sftp.
docker compose up sftp
Run test SftpIntegrationFlowsTest to produce the issue.
I can not find a solution to make it work via Docker compose file.
Finally,I added apache sshd into dependencies and prepared an EmbeddSftpServer instead.
class EmbeddedSftpServer : InitializingBean, SmartLifecycle {
private val server = SshServer.setUpDefaultServer()
@Volatile
private var port = 0
@Volatile
private var homeFolder: Path? = null
@Volatile
private var running = false
fun setPort(port: Int) {
this.port = port
}
fun setHomeFolder(path: Path) {
this.homeFolder = path
}
@Throws(Exception::class)
override fun afterPropertiesSet() {
val allowedKey: PublicKey = decodePublicKey()
server.publickeyAuthenticator =
PublickeyAuthenticator { username: String?, key: PublicKey, session: ServerSession? ->
key == allowedKey
}
server.port = port
server.keyPairProvider = SimpleGeneratorHostKeyProvider(Files.createTempFile("host_file", ".ser"))
server.subsystemFactories = Collections.singletonList(SftpSubsystemFactory())
server.fileSystemFactory = if (this.homeFolder != null) {
VirtualFileSystemFactory(this.homeFolder)
} else {
VirtualFileSystemFactory(Files.createTempDirectory("SFTP_TEMP"))
}
//server.commandFactory = ScpCommandFactory()
}
@Throws(Exception::class)
private fun decodePublicKey(): PublicKey {
val stream: InputStream = ClassPathResource("META-INF/keys/sftp_rsa.pub").inputStream
var keyBytes: ByteArray = StreamUtils.copyToByteArray(stream)
// strip any newline chars
while (keyBytes[keyBytes.size - 1].toInt() == 0x0a || keyBytes[keyBytes.size - 1].toInt() == 0x0d) {
keyBytes = Arrays.copyOf(keyBytes, keyBytes.size - 1)
}
val decodeBuffer: ByteArray = Base64.getDecoder().decode(keyBytes)
val bb: ByteBuffer = ByteBuffer.wrap(decodeBuffer)
val len: Int = bb.int
val type = ByteArray(len)
bb.get(type)
return if ("ssh-rsa" == String(type)) {
val e: BigInteger = decodeBigInt(bb)
val m: BigInteger = decodeBigInt(bb)
val spec = RSAPublicKeySpec(m, e)
KeyFactory.getInstance("RSA").generatePublic(spec)
} else {
throw IllegalArgumentException("Only supports RSA")
}
}
private fun decodeBigInt(bb: ByteBuffer): BigInteger {
val len: Int = bb.int
val bytes = ByteArray(len)
bb.get(bytes)
return BigInteger(bytes)
}
override fun isAutoStartup(): Boolean {
return PORT == port
}
override fun getPhase(): Int {
return Int.MAX_VALUE
}
override fun start() {
try {
server.start()
running = true
} catch (e: IOException) {
throw IllegalStateException(e)
}
}
override fun stop(callback: Runnable) {
stop()
callback.run()
}
override fun stop() {
if (running) {
try {
server.stop(true)
} catch (e: Exception) {
throw IllegalStateException(e)
} finally {
running = false
}
}
}
override fun isRunning(): Boolean {
return running
}
companion object {
/**
* Let OS to obtain the proper port
*/
const val PORT = 0
}
}
And declare it as bean in the test, https://github.com/hantsy/spring-puzzles/blob/master/integration-sftp/src/test/kotlin/com/example/demo/SftpIntegrationFlowsTestWithEmbeddedSftpServer.kt#L65
Check the complete example project here, https://github.com/hantsy/spring-puzzles/blob/master/integration-sftp