Having a problem with FTP transfer from our server (FileZilla, setup in passive mode) to our web application deployed on a CentOS ec2 instance. Manually testing with wget and ftp from the instance, we are able to authenticate, change directory, and fetch files. However, our ec2 deployed webapp does not fetch the files, although it DOES authenticate and navigate successfully. FTP upload in our app works fine on our LAN, and also testing externally (dev laptop on wireless tether, no VPN)
We have written more focused unit tests as part of this and are currently reproducing the failure and success cases with the following, deployed from ec2 (failure) and non-ec2 (success):
import org.junit.Test;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.net.URLConnection;
public class FTPTest {
@Test
public void testFTPAccess() throws IOException {
String fileUrl = "ftp://username:password@ftp.site.com/path/to/resource.txt";
URL url = new URL(fileUrl);
URLConnection uc = url.openConnection();
InputStream is = uc.getInputStream();
ByteArrayOutputStream out = new ByteArrayOutputStream(1024*10);
com.google.common.io.ByteStreams.copy(is,out);
System.out.println(new String(out.toByteArray()));
}
}
FileZilla server logfile is identical through login, USER, PASS, CWD, but differs once transfer begins.
Success case is as follows:
(000216) 8/28/2012 10:47:20 AM - (not logged in) (external IP)> Connected, sending welcome message...
(000216) 8/28/2012 10:47:20 AM - (not logged in) (external IP)> 220-FileZilla Server version 0.9.24 beta
(000216) 8/28/2012 10:47:20 AM - (not logged in) (external IP)> 220 Connected to server FTP!
(000216) 8/28/2012 10:47:20 AM - (not logged in) (external IP)> USER ftpUser
(000216) 8/28/2012 10:47:20 AM - (not logged in) (external IP)> 331 Password required for ftpUser
(000216) 8/28/2012 10:47:20 AM - (not logged in) (external IP)> PASS *********
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> 230 Logged on
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> TYPE I
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> 200 Type set to I
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> CWD DPS
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> 250 CWD successful. "/DPS" is current directory.
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> CWD DP276
(000216) 8/28/2012 10:47:20 AM - ftpUser (external IP)> 250 CWD successful. "/DPS/DP276" is current directory.
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> PASV ALL
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> 227 Entering Passive Mode (FTP Server IP4,190)
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> PASV
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> 227 Entering Passive Mode (FTP Server IP4,191)
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> RETR sales.txt
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> 150 Connection accepted
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> 226 Transfer OK
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> QUIT
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> 221 Goodbye
(000216) 8/28/2012 10:47:21 AM - ftpUser (external IP)> disconnected.
Failure case:
(000217) 8/28/2012 11:25:11 AM - (not logged in) (ec2 Instance IP)> Connected, sending welcome message...
(000217) 8/28/2012 11:25:11 AM - (not logged in) (ec2 Instance IP)> 220-FileZilla Server version 0.9.24 beta
(000217) 8/28/2012 11:25:11 AM - (not logged in) (ec2 Instance IP)> 220 Connected to server FTP!
(000217) 8/28/2012 11:25:11 AM - (not logged in) (ec2 Instance IP)> USER ftpUser
(000217) 8/28/2012 11:25:11 AM - (not logged in) (ec2 Instance IP)> 331 Password required for ftpUser
(000217) 8/28/2012 11:25:11 AM - (not logged in) (ec2 Instance IP)> PASS *********
(000217) 8/28/2012 11:25:11 AM - ftpUser (ec2 Instance IP)> 230 Logged on
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> TYPE I
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 200 Type set to I
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> CWD DPS
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 250 CWD successful. "/DPS" is current directory.
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> CWD DP276
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 250 CWD successful. "/DPS/DP276" is current directory.
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> EPSV ALL
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 229 Entering Extended Passive Mode (|||1318|)
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> EPSV
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 229 Entering Extended Passive Mode (|||1319|)
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> EPRT |1|ec2 internal IP|37245|
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 200 Port command successful
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> RETR products.txt
(000217) 8/28/2012 11:25:12 AM - ftpUser (ec2 Instance IP)> 150 Opening data channel for file transfer.
(000217) 8/28/2012 11:25:23 AM - ftpUser (ec2 Instance IP)> 425 Can't open data connection.
The success case uses passive mode while the failure case uses extended passive mode. You may have a hard time fine tuning the FTP connection using a plain URLConnection. I strongly suggest using a java client library instead, such as Apache Commons Net. You will find an FTP example on the home page.