Search code examples
amazon-web-servicescsvamazon-s3parquetamazon-athena

How do I Configure file format of AWS Athena results

Currently, the Athena query results are in tsv format in S3. Is there any way to configure Athena queries to return results in Parquet format.

Solution

  • Answer

    At this moment it isn't possible to do it directly with Athena. When it comes to configure result of the Athena query you can only setup query result location and encryption configuration.

    Workaround

    1) From October Athena supports CTAS query, you can try to use this feature.

    https://docs.aws.amazon.com/athena/latest/ug/ctas.html

    https://docs.aws.amazon.com/athena/latest/ug/ctas-examples.html

    I think example 4 or 5 may interest you.

    2) You can create Python script and use Boto3 framework. Here is the example, you can adapt this script to your requirements and add saving query results to the parquet format.

    https://gist.github.com/schledererj/b2e2a800998d61af2bbdd1cd50e08b76

    3) If you want to transform your data you should use rather such services as AWS EMR or AWS Glue.