Deploy to FTP via python script with GitLab CI

Question

I'm new to GitLab. I am building my first pipeline to deploy the contents of my GitLab project to an FTP server with TLS encryption. I've written a Python script using ftplib to upload the files to the FTP server that works perfectly when I run it on my local Windows machine. The script uploads the full contents of the project to a folder on the FTP server. Now I'm trying to get it to work on GitLab by calling the script in the project's .gitlab-ci.yml file. Both the script and the yml file are in the top level of my GitLab project. The setup is extremely simple for the moment:

image: python:latest

deploy:
    stage: deploy
    script: 
        - python ftpupload.py
    only:
        - main

However, the upload always times out with the following error message:

  File "/usr/local/lib/python3.9/ftplib.py", line 156, in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
  File "/usr/local/lib/python3.9/socket.py", line 843, in create_connection
    raise err
  File "/usr/local/lib/python3.9/socket.py", line 831, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
Cleaning up file based variables
ERROR: Job failed: exit code 1

Here's the basic setup for establishing the connection in the Python script that works fine locally but fails on GitLab:

class ReusedSslSocket(ssl.SSLSocket):
    def unwrap(self):
        pass


class MyFTP_TLS(ftplib.FTP_TLS):
    """Explicit FTPS, with shared TLS session"""
    def ntransfercmd(self, cmd, rest=None):
        conn, size = ftplib.FTP.ntransfercmd(self, cmd, rest)
        if self._prot_p:
            conn = self.context.wrap_socket(conn,
                                            server_hostname=self.host,
                                            session=self.sock.session)  # reuses TLS session            
            conn.__class__ = ReusedSslSocket  # we should not close reused ssl socket when file transfers finish
        return conn, size

session = MyFTP_TLS(server, username, password, timeout=None)
session.prot_p()

I know there are other tools like lftp and git-ftp that I could use in GitLab CI, but I've built a lot of custom functionality into the Python script and would like to use it. How can I successfully deploy the script within GitLab CI? Thanks in advance for your help!

Answer 1

This requires that the GitLab Runner (which executes the pipeline) is able to make an SFTP connection to your FTP server.

Shared runners are likely locked down to only connect to the GitLab server (to prevent an attack vector).

To work around this, install your own runner and register it to your GitLab.