Search code examples
pythonpython-3.xftpftplibftps

nlst times out while connecting FTPS server with Python

I can login to with Total Commander to
server: ftps://publishedprices.co.il
username: "XXXX"
password empty

And with

lftp -u XXXX: publishedprices.co.il

But when I tried to login and get the file list with Python on the same machine the nlst function returns time out.

Code:

from ftplib import FTP_TLS

ftp_server = "publishedprices.co.il"

username = 'XXXX'
password = ""

ftps = FTP_TLS()
ftps.set_debuglevel(2)
ftps.connect(ftp_server,timeout=30)
print('connected')
ftps.login(username, password)
ftps.prot_p()

print('log in')

file_list = ftps.nlst()

debug print:

*get* '220-Welcome to Public Published Prices Server\n'
*get* '220-            Created by NCR L.T.D\n'
*get* '220-\n'
*get* '220-\n'
*get* '220 ** The site is open! Have a good day.\n'
*resp* '220-Welcome to Public Published Prices Server\n220-            Created by NCR L.T.D\n220-\n220-\n220 ** The site is open! Have a good day.'
connected
*cmd* 'AUTH TLS'
*put* 'AUTH TLS\r\n'
*get* '234 Authentication method accepted\n'
*resp* '234 Authentication method accepted'
*cmd* 'USER XXXX'
*put* 'USER XXXX\r\n'
*get* '331 User XXXX, password please\n'
*resp* '331 User XXXX, password please'
*cmd* 'PASS '
*put* 'PASS \r\n'
*get* '230 Password Ok, User logged in\n'
*resp* '230 Password Ok, User logged in'
*cmd* 'PBSZ 0'
*put* 'PBSZ 0\r\n'
*get* '200 PBSZ=0\n'
*resp* '200 PBSZ=0'
*cmd* 'PROT P'
*put* 'PROT P\r\n'
*get* '200 PROT P OK, data channel will be secured\n'
*resp* '200 PROT P OK, data channel will be secured'
log in
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Type ASCII\n'
*resp* '200 Type ASCII'
*cmd* 'PASV'
*put* 'PASV\r\n'
*get* '227 Entering Passive Mode (194,90,26,21,47,54)\n'
*resp* '227 Entering Passive Mode (194,90,26,21,47,54)'

Exception:

Traceback (most recent call last):
  File "c:\project\test.py", line 18, in <module>
    file_list = ftps.nlst()
                ^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\ftplib.py", line 553, in nlst
    self.retrlines(cmd, files.append)
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\ftplib.py", line 462, in retrlines
    with self.transfercmd(cmd) as conn, \
         ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\ftplib.py", line 393, in transfercmd
    return self.ntransfercmd(cmd, rest)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\ftplib.py", line 793, in ntransfercmd
    conn, size = super().ntransfercmd(cmd, rest)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\ftplib.py", line 354, in ntransfercmd
    conn = socket.create_connection((host, port), self.timeout,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 851, in create_connection
    raise exceptions[0]
  File "C:\Users\USER\AppData\Local\Programs\Python\Python311\Lib\socket.py", line 836, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

When I set ftps.set_pasv(False) I got

*get* '220-Welcome to Public Published Prices Server\n'                                                                                                           [27/11310]*get* '220-            Created by NCR L.T.D\n'
*get* '220-\n'
*get* '220-\n'
*get* '220 ** The site is open! Have a good day.\n'
*resp* '220-Welcome to Public Published Prices Server\n220-            Created by NCR L.T.D\n220-\n220-\n220 ** The site is open! Have a good day.'
connected
*cmd* 'AUTH TLS'
*put* 'AUTH TLS\r\n'
*get* '234 Authentication method accepted\n'
*resp* '234 Authentication method accepted'
*cmd* 'USER XXXX'
*put* 'USER XXXX\r\n'
*get* '331 User XXXX, password please\n'
*resp* '331 User XXXX, password please'
*cmd* 'PASS '
*put* 'PASS \r\n'
*get* '230 Password Ok, User logged in\n'
*resp* '230 Password Ok, User logged in'
*cmd* 'PBSZ 0'
*put* 'PBSZ 0\r\n'
*get* '200 PBSZ=0\n'
*resp* '200 PBSZ=0'
*cmd* 'PROT P'
*put* 'PROT P\r\n'
*get* '200 PROT P OK, data channel will be secured\n'
*resp* '200 PROT P OK, data channel will be secured'
log in
*cmd* 'TYPE A'
*put* 'TYPE A\r\n'
*get* '200 Type ASCII\n'
*resp* '200 Type ASCII'
*cmd* 'PORT 172,17,118,200,159,233'
*put* 'PORT 172,17,118,200,159,233\r\n'
*get* '500 Port command invalid\n'
*resp* '500 Port command invalid'

lftp log:

lftp :~>debug -o log1.txt -c -t 9
lftp :~>set ftp:ssl-force true
lftp :~>set ssl:verify-certificate no
lftp :~>set ftp:use-feat false
lftp :~>connect -u XXXX: publishedprices.co.il
lftp :~>ls

log file:

2024-07-22 02:22:22 publishedprices.co.il ---- Resolving host address...
2024-07-22 02:22:22 publishedprices.co.il ---- IPv6 is not supported or configured
2024-07-22 02:22:22 publishedprices.co.il ---- 1 address found: 194.90.26.22
2024-07-22 02:22:26 publishedprices.co.il ---- Connecting to publishedprices.co.il (194.90.26.22) port 21
2024-07-22 02:22:26 publishedprices.co.il <--- 220-Welcome to Public Published Prices Server
2024-07-22 02:22:26 publishedprices.co.il <--- 220-            Created by NCR L.T.D
2024-07-22 02:22:26 publishedprices.co.il <--- 220-
2024-07-22 02:22:26 publishedprices.co.il <--- 220-
2024-07-22 02:22:26 publishedprices.co.il <--- 220 ** The site is open! Have a good day.
2024-07-22 02:22:26 publishedprices.co.il ---> AUTH TLS
2024-07-22 02:22:26 publishedprices.co.il <--- 234 Authentication method accepted
2024-07-22 02:22:26 publishedprices.co.il ---> USER XXXX
2024-07-22 02:22:26 Certificate: CN=*.publishedprices.co.il
2024-07-22 02:22:26  Issued by:        C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
2024-07-22 02:22:26  Checking against: C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
2024-07-22 02:22:26   Trusted
2024-07-22 02:22:26 Certificate: C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
2024-07-22 02:22:26  Issued by:        C=US,ST=New Jersey,L=Jersey City,O=The USERTRUST Network,CN=USERTrust RSA Certification Authority
2024-07-22 02:22:26  Checking against: C=US,ST=New Jersey,L=Jersey City,O=The USERTRUST Network,CN=USERTrust RSA Certification Authority
2024-07-22 02:22:26   Trusted
2024-07-22 02:22:26 Certificate: C=US,ST=New Jersey,L=Jersey City,O=The USERTRUST Network,CN=USERTrust RSA Certification Authority
2024-07-22 02:22:26  Issued by: C=GB,ST=Greater Manchester,L=Salford,O=Comodo CA Limited,CN=AAA Certificate Services
2024-07-22 02:22:26   Trusted
2024-07-22 02:22:26 publishedprices.co.il <--- 331 User XXXX, password please
2024-07-22 02:22:26 publishedprices.co.il ---> PASS 
2024-07-22 02:22:26 publishedprices.co.il <--- 230 Password Ok, User logged in
2024-07-22 02:22:26 publishedprices.co.il ---> PWD
2024-07-22 02:22:26 publishedprices.co.il <--- 257 "/" is the current directory
2024-07-22 02:22:26 publishedprices.co.il ---> PBSZ 0
2024-07-22 02:22:26 publishedprices.co.il <--- 200 PBSZ=0
2024-07-22 02:22:26 publishedprices.co.il ---> PROT P
2024-07-22 02:22:26 publishedprices.co.il <--- 200 PROT P OK, data channel will be secured
2024-07-22 02:22:26 publishedprices.co.il ---> PASV
2024-07-22 02:22:26 publishedprices.co.il <--- 227 Entering Passive Mode (194,90,26,21,48,206)
2024-07-22 02:22:26 publishedprices.co.il ---- Connecting data socket to (194.90.26.21) port 12494
2024-07-22 02:22:26 publishedprices.co.il ---- Data connection established
2024-07-22 02:22:26 publishedprices.co.il ---> LIST
2024-07-22 02:22:26 publishedprices.co.il <--- 150 Opening data connection
2024-07-22 02:22:26 Certificate: CN=*.publishedprices.co.il
2024-07-22 02:22:26  Issued by:        C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
2024-07-22 02:22:26  Checking against: C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
2024-07-22 02:22:26   Trusted
2024-07-22 02:22:26 Certificate: C=GB,ST=Greater Manchester,L=Salford,O=Sectigo Limited,CN=Sectigo RSA Domain Validation Secure Server CA
2024-07-22 02:22:26  Issued by:        C=US,ST=New Jersey,L=Jersey City,O=The USERTRUST Network,CN=USERTrust RSA Certification Authority
2024-07-22 02:22:26  Checking against: C=US,ST=New Jersey,L=Jersey City,O=The USERTRUST Network,CN=USERTrust RSA Certification Authority
2024-07-22 02:22:26   Trusted
2024-07-22 02:22:26 Certificate: C=US,ST=New Jersey,L=Jersey City,O=The USERTRUST Network,CN=USERTrust RSA Certification Authority
2024-07-22 02:22:26  Issued by: C=GB,ST=Greater Manchester,L=Salford,O=Comodo CA Limited,CN=AAA Certificate Services
2024-07-22 02:22:26   Trusted
2024-07-22 02:22:26 publishedprices.co.il <--- 226 Transfer complete
2024-07-22 02:22:26 publishedprices.co.il ---- Got EOF on data connection
2024-07-22 02:22:26 publishedprices.co.il ---- Closing data socket
2024-07-22 02:22:29 publishedprices.co.il ---> QUIT
2024-07-22 02:22:29 publishedprices.co.il <--- 221 Goodbye
2024-07-22 02:22:29 publishedprices.co.il ---- Closing control socket
Solution

  • The server is very unusual in using different IP address (...22) for the primary FTP port and data connections (...21).

    As it is common that servers return invalid IP addresses in PASV responses, ftplib in recent versions of Python (3.6 and newer) ignore the returned IP address and always connect to the primary address for the data connections. See Cannot list FTP directory using ftplib – but FTP client works.

    While with your unusual server, you actually want the opposite. Set FTP.trust_server_pasv_ipv4_address to make ftplib connect to the actual IP address the server returns:

    ftps.trust_server_pasv_ipv4_address = True
file_list = ftps.nlst()