Search code examples
boto3amazon-athenaapache-iceberg

How to create apache iceberg table w/ boto3


I'm trying to create an Iceberg table using boto3's glue API. Simply telling the TableType to be "ICEBERG" isn't doing it, as I still get this error when I try to run an update

NOT_SUPPORTED: Modifying Hive table rows is only supported for transactional tables

And I can confirm I'm using Athena Engine v3.

{
"Name": "<table_name_placeholder>",
"Owner": "me",
"StorageDescriptor": {
    "SerdeInfo": {
        "SerializationLibrary": "org.apache.hadoop.hive.serde2.avro.AvroSerDe",
        "Parameters": {
            "serialization.format": "1"
        }
    },
    "OutputFormat": "org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat",
    "InputFormat": "org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat",
    "Columns": <columns_placeholder>,
    "Location": "<location_placeholder>"
},
"TableType": "ICEBERG",
"Parameters": {
    "EXTERNAL": "TRUE",
    "avro.serialization.null.format": "NULL",
    "compressionType": "snappy"
},
}

Solution

  • You may have more success just using the pyiceberg library.

    Glue catalog configuration is here.