Search code examples
dockersshopensshsshdalpine-linux

How to debug a non-working passwordless RSA certificate in OpenSSH on Alpine?


I plan to write and run some integration tests using real SSH client and a SSH server. Since this requires server-level configuration, I am setting up the tests inside Docker, so a server can be built to the right spec, an OpenSSH server fired up, and the tests run.

Part of my system requires a non-root user (called nonpriv) to be able to ssh into a server on a passwordless basis. I have generated server certificates and non-root user certificates. I've set up localhost as a known host (so the server authenticity is already confirmed) but I am struggling to set up the cert as an authorised key. I want to be able to do ssh localhost as the nonpriv user and get a shell automatically. However it is skipping the key and going to password auth, which is not what I want.

The Docker ENTRYPOINT is this, so that server keys are different for every run:

#!/bin/sh
#
# With thanks to https://github.com/danielguerra69/alpine-sshd

if [ ! -f "/etc/ssh/ssh_host_rsa_key" ]; then
    # generate fresh rsa key
    ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
fi

if [ ! -f "/etc/ssh/ssh_host_dsa_key" ]; then
    # generate fresh dsa key
    ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
fi

# Prepare run dir
if [ ! -d "/var/run/sshd" ]; then
  mkdir -p /var/run/sshd
fi

# Start the SSH daemon
/usr/sbin/sshd

# Generate an SSH key for the nonpriv user
su -c /tmp/install/generate-keys.sh nonpriv

# Sleep so we can debug the server while it is running
# Will be replaced by test runner when SSH client works!
sleep 10000

As is usual for Docker, this is all run as root, so I use su -c towards the end to generate the user keys:

#!/bin/sh

mkdir -p /home/nonpriv/.ssh

# Add the server to recognised hosts
ssh-keyscan localhost >> /home/nonpriv/.ssh/known_hosts

# Generate an SSH key with an empty passphrase
ssh-keygen \
    -t rsa \
    -b 4096 \
    -f /home/nonpriv/.ssh/id_rsa \
    -N ''

# Add it to the list of authorised keys for self
ln -s /home/nonpriv/.ssh/id_rsa.pub /home/nonpriv/.ssh/authorized_keys

As you can see I use a symlink to add the public key to the authorised keys list. I then run the Docker container and shell in thusly (where "silly_name" is the automatically generated container name):

docker exec -it silly_name sh

From the shell I do this:

/ # whoami
root
/ # su nonpriv
/ $ whoami
nonpriv
/ $ ssh -vvv localhost
OpenSSH_7.4p1, LibreSSL 2.4.4
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolving "localhost" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to localhost [::1] port 22.
debug1: Connection established.
debug1: identity file /home/nonpriv/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/nonpriv/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to localhost:22 as 'nonpriv'
debug3: hostkeys_foreach: reading file "/home/nonpriv/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/nonpriv/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from localhost
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ssh-rsa SHA256:ierfDQtTWiobxAGsyEf1PrjRcmsr5jQbZVOzcNCnBo4
debug3: hostkeys_foreach: reading file "/home/nonpriv/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /home/nonpriv/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from localhost
debug1: Host 'localhost' is known and matches the RSA host key.
debug1: Found key in /home/nonpriv/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /home/nonpriv/.ssh/id_rsa (0x5645c099ae80)
debug2: key: /home/nonpriv/.ssh/id_dsa (0)
debug2: key: /home/nonpriv/.ssh/id_ecdsa (0)
debug2: key: /home/nonpriv/.ssh/id_ed25519 (0)
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: start over, passed a different list publickey,password,keyboard-interactive
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/nonpriv/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Trying private key: /home/nonpriv/.ssh/id_dsa
debug3: no such identity: /home/nonpriv/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /home/nonpriv/.ssh/id_ecdsa
debug3: no such identity: /home/nonpriv/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /home/nonpriv/.ssh/id_ed25519
debug3: no such identity: /home/nonpriv/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug3: send packet: type 50
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug3: userauth_kbdint: disable: no info_req_seen
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: 
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
nonpriv@localhost's password: 

From this it is clear that /home/nonpriv/.ssh/id_rsa is available, but it is not successful. I have tried a number of resources on the internet, often to do with permissions on the home directory, or the ~/.ssh folder, or the authorized_keys file, but to no avail. I will keep trying, but it would be useful to get some information from the system as to what the issue might be.

I think it is possible to set up keys to make ssh-ing into localhost, since I have done this in my development laptop (admittedly with Mint 18 as an OS rather than Alpine/BusyBox).

Unfortunately there are no logs at all in /var/log/, so there is not much info to go on here. Where could I look next?

Update

In case the sshd config could be important, I have the default one from Alpine:

$ less /etc/ssh/sshd_config

#   $OpenBSD: sshd_config,v 1.100 2016/08/15 12:32:04 naddy Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/bin:/usr/bin:/sbin:/usr/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile  .ssh/authorized_keys

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
#UsePAM no

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
#X11Forwarding no
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#UsePrivilegeSeparation sandbox
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
#PidFile /run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# override default of no subsystems
Subsystem   sftp    /usr/lib/ssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#   X11Forwarding no
#   AllowTcpForwarding no
#   PermitTTY no
#   ForceCommand cvs server

Solution

  • I have solved this one, though I don't fully understand why this solves it. (I will accept any answers in preference to my own if anyone has an explanation).

    In my Dockerfile I was setting up a user thus:

    # -s specify a shell; -D = don't prompt for a password
    RUN adduser -s /bin/sh -D nonpriv
    

    As far as I know that is just a user with a home directory and a null password. However, it looks like I do in fact need to specify a password, so I now also do this:

    # It looks like passwordless access does not work unless the user
    # has a password!
    RUN echo 'nonpriv:Password123' | chpasswd
    

    That seems odd to me, since the PPK access system should not care what the password of the user is, or whether it has one.

    I can now SSH into self:

    / $ whoami
    nonpriv
    / $ ssh localhost
    Welcome to Alpine!
    
    The Alpine Wiki contains a large amount of how-to guides and general
    information about administrating Alpine systems.
    See <http://wiki.alpinelinux.org>.
    
    You can setup the system with the command: setup-alpine
    
    You may change this message by editing /etc/motd.
    
    d4dded05c2d1:~$