AWS Data Wrangler - wr.athena.read_sql_query doesn't work

Question

I started using AWS Data Wrangler lib ( https://aws-data-wrangler.readthedocs.io/en/stable/what.html )

to execute queries on AWS Athena and use the results of them in my AWS Glue python shell job.

I see that exist wr.athena.read_sql_query to obtain what I need.

This is my code:

import sys
import os
import awswrangler as wr

os.environ['AWS_DEFAULT_REGION'] = 'eu-west-1'
databases = wr.catalog.databases()
print(databases)

query='select count(*) from staging_dim_channel'
print(query)
df_res = wr.athena.read_sql_query(sql=query, database="lsk2-target")
print(df_res)
print(f'DataScannedInBytes:            {df_res.query_metadata["Statistics"]["DataScannedInBytes"]}')
print(f'TotalExecutionTimeInMillis:    {df_res.query_metadata["Statistics"]["TotalExecutionTimeInMillis"]}')
print(f'QueryQueueTimeInMillis:        {df_res.query_metadata["Statistics"]["QueryQueueTimeInMillis"]}')
print(f'QueryPlanningTimeInMillis:     {df_res.query_metadata["Statistics"]["QueryPlanningTimeInMillis"]}')
print(f'ServiceProcessingTimeInMillis: {df_res.query_metadata["Statistics"]["ServiceProcessingTimeInMillis"]}')

I retrieve without problem the list of database (including the lsk2-target), but the read_sql_query go on error and I receive:

WaiterError: Waiter BucketExists failed: Max attempts exceeded

Please, can you help me to understand where I am wrong? Thanks!

Answer 1

I solved using Boto3 for Athena