I have some data in S3, created a schema in the Glue catalog, and then exposed it to QuickSight via Athena. All this works great when I create it by clicking in the console.
I then converted it to the following CloudFormation:
AnalyticsDatabase:
Type: AWS::Glue::Database
Properties:
DatabaseInput:
Name: analytics
CatalogId: !Ref AWS::AccountId
RawAnalysisAnalyticsTable:
Type: AWS::Glue::Table
Properties:
DatabaseName: !Ref AnalyticsDatabase
CatalogId: !Ref AWS::AccountId
TableInput:
Name: analysis_raw
TableType: EXTERNAL_TABLE
Parameters:
classification: json
StorageDescriptor:
Columns:
- {Name: id, Type: string}
- {Name: treeid, Type: string}
- {Name: patientid, Type: string}
Compressed: false
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
Location: s3://my-bucket/dynamodb/Analysis/
NumberOfBuckets: 0
SerdeInfo:
Parameters: {paths: 'id,patientid,treeid'}
SerializationLibrary: org.openx.data.jsonserde.JsonSerDe
SortColumns: []
StoredAsSubDirectories: false
However, when I try to pull the CF-created table into QuickSight I get:
Your database generated a SQL exception. This can be caused by query timeouts, resource constraints, unexpected DDL alterations before or during a query, and other database errors. Check your database settings and your query, and try again.
region: us-east-1
timestamp: 1544113019756
requestId: 5ab8f9a2-f972-11e8-b201-154c30728c75
sourceErrorCode: 0
sourceErrorMessage: [Simba][JDBC](11380) Null pointer exception.
sourceErrorState: HY000
sourceException: java.sql.SQLException
sourceType: ATHENA
Does anyone have any idea what this error means or how I can troubleshoot it? I've compared all the properties of the manually-created table to the CloudFormation-created table, and they seem identical.
Max's answer should be the accepted answer here. Replicated this and only solution that worked was to add the PartitionKeys: [] parameter. I had initially added it as a child or StorageDescription, which didn't work. Has to be added at the TableInput child level as specified in the docs. It is the right answer because none of the other conditions listed here (security, etc) will give the NullPointerException that is referenced in the question.