Trying to transfer a file from one remote host to another, via sftp.
sftp dependencies:
implementation 'org.apache.commons:commons-vfs2:2.4'
compile 'com.jcraft:jsch:0.1.55'
running the code in an AWS EKS Kubernetes pod with access to both remote hosts(as the connection is successful). facing the following error in Kubernetes pod:
LOG
o.a.c.v.provider.sftp.SftpClientFactory : Connecting to host port 22
o.a.c.v.provider.sftp.SftpClientFactory : Connection established
o.a.c.v.provider.sftp.SftpClientFactory : Remote version string: SSH-2.0-JSCAPE
o.a.c.v.provider.sftp.SftpClientFactory : Local version string: SSH-2.0-JSCH-0.1.54
o.a.c.v.provider.sftp.SftpClientFactory : CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256
o.a.c.v.provider.sftp.SftpClientFactory : CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
o.a.c.v.provider.sftp.SftpClientFactory : CheckSignatures: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_KEXINIT sent
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_KEXINIT received
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: ssh-rsa,rsa-sha2-256,rsa-sha2-512
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: aes256-ctr,aes192-ctr,aes128-ctr
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: aes256-ctr,aes192-ctr,aes128-ctr
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: hmac-sha1,hmac-sha2-512
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: hmac-sha1,hmac-sha2-512
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: none
o.a.c.v.provider.sftp.SftpClientFactory : kex: server: none
o.a.c.v.provider.sftp.SftpClientFactory : kex: server:
o.a.c.v.provider.sftp.SftpClientFactory : kex: server:
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: none
o.a.c.v.provider.sftp.SftpClientFactory : kex: client: none
o.a.c.v.provider.sftp.SftpClientFactory : kex: client:
o.a.c.v.provider.sftp.SftpClientFactory : kex: client:
o.a.c.v.provider.sftp.SftpClientFactory : kex: server->client aes128-ctr hmac-sha1 none
o.a.c.v.provider.sftp.SftpClientFactory : kex: client->server aes128-ctr hmac-sha1 none
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_KEX_DH_GEX_REQUEST(1024<2048<2048) sent
o.a.c.v.provider.sftp.SftpClientFactory : expecting SSH_MSG_KEX_DH_GEX_GROUP
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_KEX_DH_GEX_INIT sent
o.a.c.v.provider.sftp.SftpClientFactory : expecting SSH_MSG_KEX_DH_GEX_REPLY
o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
o.s.web.servlet.DispatcherServlet : Completed initialization in 47 ms
o.a.c.v.provider.sftp.SftpClientFactory : ssh_rsa_verify: signature true
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_NEWKEYS sent
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_NEWKEYS received
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_SERVICE_REQUEST sent
o.a.c.v.provider.sftp.SftpClientFactory : SSH_MSG_SERVICE_ACCEPT received
o.a.c.v.provider.sftp.SftpClientFactory : Authentications that can continue: publickey,keyboard-interactive,password
o.a.c.v.provider.sftp.SftpClientFactory : Next authentication method: publickey
o.a.c.v.provider.sftp.SftpClientFactory : Authentications that can continue: password
o.a.c.v.provider.sftp.SftpClientFactory : Next authentication method: password
o.a.c.v.provider.sftp.SftpClientFactory : Authentication succeeded (password).
m.j.r.utils.helper.FileTransferHelper : FileTransferFailed due to IOException:
org.apache.commons.vfs2.FileSystemException: Could not copy "sftp://username:***@host/fileWithPath" to "file:///tmp/ImpFile".
at org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:300) ~[commons-vfs2-2.4.jar:2.4]
at my.package.name.utils.helper.FileTransferHelper.transferFile(FileTransferHelper.java:39) ~[classes/:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:305) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:190) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:153) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360) ~[spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE]
at org.springframework.boot.context.event.EventPublishingRunListener.running(EventPublishingRunListener.java:103) ~[spring-boot-2.2.5.RELEASE.jar:2.2.5.RELEASE]
at org.springframework.boot.SpringApplicationRunListeners.running(SpringApplicationRunListeners.java:77) ~[spring-boot-2.2.5.RELEASE.jar:2.2.5.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:330) ~[spring-boot-2.2.5.RELEASE.jar:2.2.5.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) ~[spring-boot-2.2.5.RELEASE.jar:2.2.5.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) ~[spring-boot-2.2.5.RELEASE.jar:2.2.5.RELEASE]
at money.jupiter.reporting.ReportingApplication.main(ReportingApplication.java:17) ~[classes/:na]
Caused by: java.io.IOException: error
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1448) ~[jsch-0.1.55.jar:na]
at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:290) ~[na:na]
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351) ~[na:na]
at org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:106) ~[commons-vfs2-2.4.jar:2.4]
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:252) ~[na:na]
at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:292) ~[na:na]
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:351) ~[na:na]
at org.apache.commons.vfs2.util.MonitorInputStream.read(MonitorInputStream.java:106) ~[commons-vfs2-2.4.jar:2.4]
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:107) ~[na:na]
at org.apache.commons.vfs2.provider.DefaultFileContent.write(DefaultFileContent.java:804) ~[commons-vfs2-2.4.jar:2.4]
at org.apache.commons.vfs2.provider.DefaultFileContent.write(DefaultFileContent.java:784) ~[commons-vfs2-2.4.jar:2.4]
at org.apache.commons.vfs2.provider.DefaultFileContent.write(DefaultFileContent.java:755) ~[commons-vfs2-2.4.jar:2.4]
at org.apache.commons.vfs2.provider.DefaultFileContent.write(DefaultFileContent.java:771) ~[commons-vfs2-2.4.jar:2.4]
at org.apache.commons.vfs2.FileUtil.copyContent(FileUtil.java:37) ~[commons-vfs2-2.4.jar:2.4]
at org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:295) ~[commons-vfs2-2.4.jar:2.4]
... 22 common frames omitted
Code
private static void transferFile(String sourceFile, String destinationFile, boolean overWrite) {
try {
var fileSystemManager = VFS.getManager();
try (var destinationFileObject = fileSystemManager.resolveFile(destinationFile)) {
try (var sourceFileObject = fileSystemManager.resolveFile(sourceFile)) {
if (!overWrite && destinationFileObject.exists()) {
log.error("file already exists and overWrite is disabled");
throw new ReportingException(FILE_ALREADY_EXISTS);
}
destinationFileObject.copyFrom(sourceFileObject, Selectors.SELECT_SELF);
}
}
} catch (IOException ex) {
log.error("FileTransferFailed due to IOException:", ex);
throw new ReportingException(FILE_TRANSFER_FAILED);
}
}
}
Based on this line(ChannelSftp.java:1448) from error looked at the "ChannelSftp" file, and that found this error is thrown when "header.type" is set to 101(SSH_FXP_STATUS), but found no lead after that.
(I was able to download the file available in the free SFTP server when running from my laptop, but it's giving me this issue in my org's beta system LINK of free SFTP server)
I tried performing readable check before copy with "if(sourceFileObject.isReadable())", but the execution stops at this line and neither continues nor shows timeout.
Any help is appriciated.
I implemented the code using a different library(used just jcraft:jsch instead of vfs2) and got a better error message: "error:3 permission denied", found that the user had the permission just to view and list files and no permission to download. changing the user to one with the download permission resolved my issue.