I want to make sure that the only way to access a computer through SSH is if the client's key is already added to the authorized_keys file on the server.
For this, I edit /etc/ssh/sshd_config and enable the following options
ChallengeResponseAuthentication no
PasswordAuthentication no
PubkeyAuthentication yes
UsePAM no
Following this, I restart the sshd service
sudo /etc/init.d/ssh restart
However, when I try to ssh into the server from a client whose key is not saved in the server's authorized keys file, I get prompted for a password.
EDIT: I have figured out what the problem is, but I am going to leave this question here in case it is useful to someone else.
What happens is that when connecting from a client with a key not in the authorized_keys file, the server asks for the password, but no matter what password is entered, it is declined. I think this is so that it is difficult for attackers to understand how the sshd is configured.
The behavior that I described above is the intended behavior of the sshd. No matter what password is entered, it will not be accepted.