As the "ec2-user", we can SSH and SCP to our EC2 instance just fine.
As any other user, we can SSH just fine, but cannot SCP.
Instead, when trying to SCP as any user other than "ec2-user", our ec2 instance spawns a new (dup) ssh-agent and does not receive files.
--
We use ssh-agent and each user ssh-add their own .pem key. Their public key is added on the remote ec2 instance, in each user's ~/.ssh/authorized_keys. Again, our config works fine for SSH.
The contents of (local) /home/jonathan/.ssh/config is:
Host my-ec2.com
User jdoe
Hostname 123.123.123.123
ForwardAgent yes
IdentityFile /Users/jonathan/.ssh/key.pem
And there is no remote ~/.ssh/config on ec2, we just used the system defaults.
I'm really not sure why the local and/or remote ssh-agent's work differently with SSH than they do with SCP.
Any help would be appreciated!!
DEBUG INFO
This works -- ssh as "ec2-user":
$> ssh-add /Users/jonathan/.ssh/key.pem
$> ssh [email protected]
This works too -- ssh as "jdoe":
$> ssh-add /Users/jonathan/.ssh/key.pem
$> ssh [email protected]
This also works -- scp as "ec2-user":
$> ssh-add /Users/jonathan/.ssh/key.pem
$> scp foo.txt [email protected]:~/.
foo.txt 100% 197 2.5KB/s 00:00
This does NOT work -- scp as "jdoe":
$> ssh-add /Users/jonathan/.ssh/key.pem
$> scp foo.txt [email protected]:~/.
Agent PID 12345
Additionally, every failed scp attempt creates a duplicate ssh-agent on ec2
$> ssh-add /Users/jonathan/.ssh/key.pem
$> ssh [email protected]
$> ps -x | grep ssh-agent
11677 ? Ss 0:00 ssh-agent
11708 ? Ss 0:00 ssh-agent
11742 ? Ss 0:00 ssh-agent
11919 ? Ss 0:00 ssh-agent
12345 ? Ss 0:00 ssh-agent
### a duplicate copy if ssh-agent is running with PID 12345
### there are as many ssh-agent running as failed SCP attempts...
Here's a copy of the full "scp -v" output:
$> scp -v foo.txt [email protected]:~/.
Executing: program /usr/bin/ssh host my-ec2.com, user jdoe, command scp -v -t ~/foo
OpenSSH_7.4p1, LibreSSL 2.5.0
debug1: Reading configuration data /Users/jonathan/.ssh/config
debug1: /Users/jonathan/.ssh/config line 2: Applying options for *
debug1: /Users/jonathan/.ssh/config line 7: Applying options for my-ec2.com
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 123.123.123.123 [123.123.123.123] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /Users/jonathan/.ssh/cps-keypair.pem type -1
debug1: key_load_public: No such file or directory
debug1: identity file /Users/jonathan/.ssh/cps-keypair.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 123.123.123.123:22 as 'jdoe'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:j4kSCIWNUz5k7LHHK+n9iR9kxktihrD4X/srX4uX/5U
debug1: Host '123.123.123.123' is known and matches the ECDSA host key.
debug1: Found key in /Users/jonathan/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/jonathan/.ssh/cps-keypair.pem
debug1: Server accepts key: pkalg rsa-sha2-512 blen 279
debug1: Authentication succeeded (publickey).
Authenticated to 123.123.123.123 ([123.123.123.123]:22).
debug1: channel 0: new [client-session]
debug1: Requesting [email protected]
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype [email protected] want_reply 0
debug1: Sending command: scp -v -t ~/foo
Agent pid 24732
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
Transferred: sent 2812, received 2696 bytes, in 0.2 seconds
Bytes per second: sent 15501.9, received 14862.4
debug1: Exit status 0
Start ssh-agent on your EC2 instance only for interactive sessions. You will not need it for scp, and it's output causes your scp to fail.
Apparently ssh-agent gets called during login on your EC2 instance (e.g. in your Bash profile).
To prevent it from starting, adjust your profile script on your EC2 instance - e.g. by adding a if statement like this one (assuming you're using Bash on your EC2 instance):
if [[ "$-" == *i* ]]; then
# <start ssh agent here>
fi