Can you ingest an encrypted S3 Object (Text file) using AWS Glue?

Question

I have an encrypted text file in S3. I want to load this data to RDS Aurora using AWS Glue.

I created a crawler to map the schema of my encrypted text file but crawler doesn't create any table. Just to be sure, I crawler the same file but on a decrypted version and it works.

My question is:

1. Can't AWS Glue directly ingest an encrypted file from S3?
2. If yes, please let me know what is it that I'm missing?
3. If no, what the best way to ingest encrypted data into AWS Glue without decrypting it at all. (My data is highly sensitive)

Thanks in advance.

Answer 1

AWS Glue doesn't have the ability to ingest the encrypted file if it's encrypted using a client side encryption. However, if you upload a file to S3 data store using a server side encryption, AWS glue can ingest this data provided you give policy access to the role that allows the crawler to run and access your Amazon S3 data stores.

Here is the policy that I defined:

{
    "Version": "2012-10-17",
    "Statement": {
        "Effect": "Allow",
        "Action": [
            "kms:Decrypt"
        ],
        "Resource": [
            "arn:aws:kms:us-east-1:<Your Account Number>:key/<Your Key ID>"
        ]
    }
}

So, the trick is to enable the server side encryption for your data and not the client side encryption.