I have been looking at attr_encrypted, but it stores the keys in the code, which doesn't seem to be so secure. If my webserver gets breached, the encryption won't help much.
What are some options for which the data could still be secure if my webserver gets breached?
The keys have to be in memory in order to perform encryption and decryption. If your server is breached, you're pooched no matter how you store your keys; once an attacker has access to your local machine, all bets are off. If your app can decrypt data, they'll be able to, as well, depending on how much effort they're willing to spend to extract those keys.
attr_encrypted will protect you against things like SQL injections resulting in sensitive data being leaked, but if your app is compromised, then your attacker can get to anything your app can get to.
To make it harder, though, you can use password-encrypted keys. You can store your encrypted key on disk, and then when you deploy your app (or otherwise start it up), you will provide your password to decrypt the key into memory, then throw the password away. This still can't protect you against someone grepping through your process's memory, and it means that you have to have human interaction every time your app needs to start up (which could be a problem with automated monitoring) but it would substantially increase the difficulty of (but not prevent) extracting encryption keys from a compromised machine.