I am new to AWS. Both the application and the database being used reside in AWS servers. Say, to protect the data at rest, I encrypt my database using KMS service. But the application will have access to the key, because the application has to access the database to perform operations. So doesn't it imply that key also resides in AWS servers. Doesn't this in turn pose threat that Amazon can access this key or rather any other customer of AWS can hack this key.
There are a few elements I will address first from your post. It seems from your title you have application code on a EC2 server that is using a database in RDS. I am going to assume your EC2 instance is using EBS for storage.
But the application will have access to the key, because the application has to access the database to perform operations.
Technically, the EC2 instance will not have access to the underlying data store in RDS. Even if you use the same master key in KMS, a different data key will be used to encrypt the EC2 instance EBS volume and RDS volume. Now, of course, the RDS instance has a standard data connection that allows access with proper credentials, but that is different than saying I can 'grep' or change the underlying database files/content. This is just the standard connection permitted by whichever database you are using to make queries to it and return responses. When RDS saves data to non-volatile memory, the underlying storage will be encrypted.
So doesn't it imply that key also resides in AWS servers.
Absolutely, yes. In order for the PaaS (Platform as a Service) to work with encrypted underlying data, it needs access to the encryption key to read the data.
Using AWS, like everything, is a risk. You (and presumably your organization) need to make informed decisions about the level of risk you are willing to take and weigh against what you are using it for (and the level of sensitivity of the data you are exposing).
With respect to encryption, AWS has a number of different levels of control that it allows customers to exercise. I will generally list them below from loose to tight. Of course, the more control you exert, the more responsibility (danger) falls on your side of the equation.
A) The simplest, and typically default, offered by AWS is using amazon managed keys. These are the ones that look like aws/ebs or aws/rds. In this case, you are using keys that Amazon creates and fully manages. However, there is no auditing of use of the specific keys since the master key is shared. (Access to the resources that use it of course, are still audited with CloudTrail to the extent that service publishes to CloudTrail). With KMS, envelope encryption is used. This means that although the 'master' key may be the same, the master key is just encrypting another encryption key which is unique. For example, two different EBS volumes created with the same master key will have different underlying encryption keys associated with the data.
B) Next, you can move to CMKs. This is where you go into KMS and specifically create a master key (Customer Master Key) for your use. There is a monthly charge for each master key, although it can encrypt countless underlying keys underneath it. If you create it with amazon created material, this means that AWS will create the key (including the underlying cryto key). It will never allow you to see the underlying crypto key, which may be a plus or minus for you. It can be configured to automatically rotate the key material on an annual basis if you prefer. The CMK is specifically for you and your account. By default, it will allow IAM policies to control access to the key for your account only. It can be an effective defense in depth protection. For example, if someone mistakenly gave a S3 bucket policy that allowed public access to the data, by default, KMS would only permit access to the key to authenticated users delegated permission to the CMK and effectively deny access to the data in the bucket to public users. This of course assumes the data was encrypted in the first place. Finally, every operation that uses that CMK will be audited with CloudTrail. This lets you inspect how and where the encryption key is being used. Here is an excerpt from the security FAQ for KMS:
AWS KMS is designed so that no one, including AWS employees, can retrieve your plaintext CMKs from the service. AWS KMS uses hardware security modules (HSMs) that have been validated under FIPS 140-2, or are in the process of being validated, to protect the confidentiality and integrity of your keys. Your keys are stored in these HSMs regardless of whether you use AWS KMS or AWS CloudHSM to create your keys or you import key material into a CMK. Your plaintext CMKs never leave the HSMs, are never written to disk, and are only ever used in the volatile memory of the HSMs for the time needed to perform your requested cryptographic operation. Updates to software on the service hosts and to the AWS KMS HSM firmware is controlled by multi-party access control that is audited and reviewed by an independent group within Amazon and a NIST-certified lab in compliance with FIPS 140-2.
C) Next, you can use CMKs, but decide to not trust Amazon to create the crypto key material. You transport the key material securely, in a somewhat complicated, but necessary process to ensure the integrity and confidentiality of the key material. When you create the CMK, you create a 'shell' of a key that is ready to import the key material. Until you do so, the key can't be used in any rational form.
Generally, this has the same pros/cons with Amazon generated CMKs. However, there are two notable exceptions: 1) recovery - even though the CMKs are redundantly backed up at AWS throughout the selected region, in the event of a prolonged power event (or some other disaster) and AWS lost access to the key material you provided, it has no way of restoring the key material. You will need to supply the key material again to load into the CMK. There is a secure method to do this, but now you have to carefully control and protect that key material on your side. Your ability to do this securely has probably the largest effect on the security of the underlying data. 2) Since AWS is assuming you are controlling the key material, it will permit you to immediately purge a CMK. In B), since the CMK could control dozens, hundreds, thousands, or even countless data elements, destroying the key is a disaster if it was a mistake. To protect against this, AWS will not let you immediately delete a key ever. It will instead schedule the key for deletion and immediately suspend use of the key. Under no circumstances can the deletion be faster than 7 days. However, if you want the ability to immediately clear AWS of the crypto behind the key CMKs with customer supplied content is the way to go.
D) If you want even more control over how encryption keys are managed in AWS you can use CloudHSM. This is effectively your own dedicated key server in AWS. However, you lose significant things here. Although you have a lot of control, the service is expensive, and many (almost all) PaaS offerings do not work with CloudHSM (for example: RDS, although I think Oracle RDS works). I should note that this service was relatively early in the KMS offerings, particularly before they made some other guarantees about the KMS keys created (like FIPS certification). As such, it isn't as well integrated with the other services and seems to feel like a red-headed step-child in the AWS offerings. I get the impression this is a niche application and it doesn't enjoy a lot of love from AWS. This is a 'go it alone' offering.
E) Instead of using server side encryption (where AWS has access to the underlying encryption key in storage somewhere), you can use client-side encryption. Fewer AWS services support this (the only one I am aware of is S3). This is like C) in that you need to provide security for all the encryption keys and provide them whenever underlying data access is needed.
F) Totally off the grid would encrypt it before it ever goes to AWS. Of course, this precludes you using the data in AWS, like in an EC2 instance or RDS instance, since AWS couldn't decode or use the data.
At the end of the day, you need to decide what is worth the risk and what isn't. How are you are your organization at creating secure configurations for the services you are considering using on AWS versus on-prem? How does the cost of that weigh against the potential for control/compromise on AWS's part? The answers to these will ultimately determine if AWS is for you.