I'm trying to understand public and private keys, instead of just using them. I understand that both the public and private key have to create a pair, but if the server you want access to is receiving a decrypted private key, I don't see how it's better than a password.
Private keys are never supposed to be given to anyone, that's why it is called a private key. The private key is only known by the computer that uses it.
When a client wants to send an encrypted message to a server, it first requests the server's public key. It uses this key to encrypt the message, then sends it to the server. The server uses its private key to decrypt it.
When the key is known by both the server and the client ahead of time, the security is more based on the complexity of the algorithm than the key itself, so a simple password could easily get someone very far (though, many single-key encryption algorithms require larger keys to be secure against a computer).
However, if two keys are used, with one of them having the purpose of being used by everyone, and the other having the purpose of being the only one that can decrypt, the two keys must be related and the encryption algorithm must be exactly the same as what everyone else is using. This means a hacker can observe the public key, and the encrypted message to try and find the private key, so they can decrypt the message.
To combat this, we specially design these keys to have difficult to find relationships. This makes it so that even with many computers, unless the hacker got extremely lucky, the hacker would be dead for many generations before they could decrypt it (many people say billions of years, but I think their exaggerating or taking the best-case scenario).
https://en.m.wikibooks.org/wiki/Cryptography/A_Basic_Public_Key_Example