In my understanding, when the local machine try to connect to a remote server, the private key stored in ./ssh/id_rsa is used and the corresponding public key stored in ./ssh/id_rsa.pub is sent to the server. If the remote server find a match of this public key in the authorized_key file, the authentication begins. But with Jenkin's SSH Agent plugin, only the private key is provided through the Username and SSH private key credential. A public key was never provided in any way. How does the server know which public key it should use to do the authentication then?
My hypothesis is that the server associate each username with the public key stored in authorized_key file? But honestly, it doesn't seem to be the case though.
The basic idea of a public key cryptosystem is that the public key can be easily derived from the private key
https://www.ssh.com/academy/cryptography/private-and-public-keys
Easy to try by running this command
ssh-keygen -y