I have a bash function that's called during a GitHub Workflow run. I'm using Sftp to change permissions on a bunch of remote files — without shell access Sftp is the only way available to me to change the permission using CHMOD.
When I run it in a Docker container with a specific version of Ubuntu and Expect — it runs perfectly. However, with the same OS environment and Expect — it fails in a GitHub Workflow.
build-tool.shftp_put () {
local FILE_PERM=${1} \
FILE_PATH=${2} \
FTP_HOST=${3} \
FTP_USER=${4} \
FTP_PWD=${5}
printf "\n🔒 Setting %s permissions for %s ...\n" "$FILE_PERM" "$FILE_PATH"
# Connect to the FTP server over
expect << EOF
spawn sftp -v $FTP_USER@$FTP_HOST
expect "*yes/no*" {
set timeout 30
send -- "yes\r";
exp_continue
}
expect "*?assword:*" {
set timeout 30
send -- "$FTP_PWD\r"
}
expect "Permission denied, please try again." {
exit 1;
exp_continue
}
expect "sftp>*" {
send -- "cd $FILE_PATH\r"
}
expect "sftp>*" {
send -- "chmod $FILE_PERM .\r"
}
expect "sftp>*" {
set timeout 60
send -- "quit\n"
}
expect eof
EOF
}
...
Note: SFTP is being called with the verbose -v option, which gets passed to SSH.
Locally, the script is called using remote FTP details which are stored in environment variables.
. ./build-tool.sh
ftp_put "$FILE_PERM" "$FILE_PATH" "$FTP_HOST" "$FTP_USER" "$FTP_PWD"
✅ The script successfully works, and applies the required file permissions:
🔒 Setting 555 permissions for app/sprinkles ...
spawn sftp -v [email protected]
OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to ssh.mywebsite.com [46.30.211.80] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.1
debug1: Remote protocol version 2.0, remote software version OneSSH-Proxy_1
debug1: compat_banner: no match: OneSSH-Proxy_1
debug1: Authenticating to ssh.mywebsite.com:22 as 'mywebsite.com'
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:8sBBlU4Q4RqUzZO1J0RpAucj6/DicaJI1TjPDcRO22U
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'ssh.mywebsite.com' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: /root/.ssh/id_rsa
debug1: Will attempt key: /root/.ssh/id_ecdsa
debug1: Will attempt key: /root/.ssh/id_ecdsa_sk
debug1: Will attempt key: /root/.ssh/id_ed25519
debug1: Will attempt key: /root/.ssh/id_ed25519_sk
debug1: Will attempt key: /root/.ssh/id_xmss
debug1: Will attempt key: /root/.ssh/id_dsa
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password,publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ecdsa_sk
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: Trying private key: /root/.ssh/id_ed25519_sk
debug1: Trying private key: /root/.ssh/id_xmss
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Next authentication method: password
[email protected]'s password:
Authenticated to ssh.mywebsite.com ([46.30.211.80]:22) using "password".
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: Sending environment.
debug1: Sending subsystem: sftp
debug1: Using server download size 261120
debug1: Using server upload size 261120
debug1: Server handle limit 1048571; using 64
Connected to ssh.mywebsite.com.
sftp> cd app/sprinkles
sftp> chmod 555 .
Changing mode on /customers/a/e/9/mywebsite.com/httpd.www/app/sprinkles/.
sftp> quit
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2516, received 2236 bytes, in 29.4 seconds
Bytes per second: sent 85.6, received 76.0
debug1: Exit status -1
In GitHub, the workflow uses remote FTP server details which are stored in GitHub secrets. Other parts of the workflow use the values in these secrets — therefore the issue here isn't with the FTP password being incorrect.
- name: "FTP: 🔏 Add write permissions"
run: |
. ./build-tool.sh
ftp_put \
755 \
app/sprinkles \
${{ secrets.FTP_PROD_SERVER }} \
${{ secrets.FTP_PROD_USERNAME }} \
${{ secrets.FTP_PROD_PASSWORD }}
...
Note: I've tried putting the secrets in double quotes, and also to call ftp_put in a single line, without success.
❌ The script fails with:
spawn sftp -v ***@***
OpenSSH_8.9p1 Ubuntu-3ubuntu0.1, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to *** [***] port 22.
debug1: Connection established.
debug1: identity file /home/runner/.ssh/id_rsa type -1
debug1: identity file /home/runner/.ssh/id_rsa-cert type -1
debug1: identity file /home/runner/.ssh/id_ecdsa type -1
debug1: identity file /home/runner/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/runner/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/runner/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/runner/.ssh/id_ed25519 type -1
debug1: identity file /home/runner/.ssh/id_ed25519-cert type -1
debug1: identity file /home/runner/.ssh/id_ed25519_sk type -1
debug1: identity file /home/runner/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/runner/.ssh/id_xmss type -1
debug1: identity file /home/runner/.ssh/id_xmss-cert type -1
debug1: identity file /home/runner/.ssh/id_dsa type -1
debug1: identity file /home/runner/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.1
debug1: Remote protocol version 2.0, remote software version OneSSH-Proxy_1
debug1: compat_banner: no match: OneSSH-Proxy_1
debug1: Authenticating to ***:22 as '***'
debug1: load_hostkeys: fopen /home/runner/.ssh/known_hosts: No such file or directory
debug1: load_hostkeys: fopen /home/runner/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:8sBBlU4Q4RqUzZO1J0RpAucj6/DicaJI1TjPDcRO22U
debug1: load_hostkeys: fopen /home/runner/.ssh/known_hosts: No such file or directory
debug1: load_hostkeys: fopen /home/runner/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: hostkeys_find_by_key_hostfile: hostkeys file /home/runner/.ssh/known_hosts does not exist
debug1: hostkeys_find_by_key_hostfile: hostkeys file /home/runner/.ssh/known_hosts2 does not exist
debug1: hostkeys_find_by_key_hostfile: hostkeys file /etc/ssh/ssh_known_hosts2 does not exist
The authenticity of host '*** (***)' can't be established.
ECDSA key fingerprint is SHA256:8sBBlU4Q4RqUzZO1J0RpAucj6/DicaJI1TjPDcRO22U.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
debug1: SELinux support disabled
Warning: Permanently added '***' (ECDSA) to the list of known hosts.
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /home/runner/.ssh/id_rsa
debug1: Will attempt key: /home/runner/.ssh/id_ecdsa
debug1: Will attempt key: /home/runner/.ssh/id_ecdsa_sk
debug1: Will attempt key: /home/runner/.ssh/id_ed25519
debug1: Will attempt key: /home/runner/.ssh/id_ed25519_sk
debug1: Will attempt key: /home/runner/.ssh/id_xmss
debug1: Will attempt key: /home/runner/.ssh/id_dsa
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,[email protected],[email protected],ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-256,rsa-sha2-512,ssh-rsa,ssh-dss>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password,publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/runner/.ssh/id_rsa
debug1: Trying private key: /home/runner/.ssh/id_ecdsa
debug1: Trying private key: /home/runner/.ssh/id_ecdsa_sk
debug1: Trying private key: /home/runner/.ssh/id_ed25519
debug1: Trying private key: /home/runner/.ssh/id_ed25519_sk
debug1: Trying private key: /home/runner/.ssh/id_xmss
debug1: Trying private key: /home/runner/.ssh/id_dsa
debug1: Next authentication method: password
***@***'s password:
debug1: Authentications that can continue: password,publickey
Permission denied, please try again.
What's going wrong here?
After echo'ing the GitHub secrets to the workflow output, with e.g. echo ${{ secrets.FTP_PROD_PASSWORD }} | sed 's/./& /g', I found that the FTP account that I'm using in the rest of the workflow is using regular non-encrypted FTP (Port 21), whereas the SFTP from my Docker container is using a SSH/SFTP login (Port 22). Problem solved! In hindsight, a simple mix-up.
Works perfectly.
🔒 Setting 755 permissions for app/sprinkles ...
spawn sftp ***@***
The authenticity of host '*** (***)' can't be established.
ECDSA key fingerprint is SHA256:**********/********.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '***' (ECDSA) to the list of known hosts.
***@***'s password:
Connected to ***.
sftp> cd app/sprinkles
sftp> chmod 755 .
Changing mode on /customers/a/e/9/***/httpd.www/app/sprinkles/.
sftp> quit
🔒 Setting 755 permissions for app/vendor ...
spawn sftp ***@***
***@***'s password:
Connected to ***.
sftp> cd app/vendor
sftp> chmod 755 .
Changing mode on /customers/a/e/9/***/httpd.www/app/vendor/.
sftp> quit
...