Search code examples
hivedelta-lake

Can't create a delta lake table in hive

I'm trying to create a delta lake table in hive 3. I moved the delta-hive-assembly_2.11-0.3.0.jar to the hive aux directory and set into the hive cli

SET hive.input.format=io.delta.hive.HiveInputFormat;
SET hive.tez.input.format=io.delta.hive.HiveInputFormat;

but when I try to create the table it throws the following error:

[2b8497c1-b4d3-492e-80a5-ec4db4119018 HiveServer2-Handler-Pool: Thread-133]: Exception occured while getting the URI from storage handler: Expected authority at index 22: deltastoragehandler://: Unsupported ex
java.net.URISyntaxException: Expected authority at index 22: deltastoragehandler://: Unsupported ex
    at org.apache.hadoop.hive.ql.metadata.DefaultStorageHandler.getURIForAuth(DefaultStorageHandler.java:76) ~[hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]
    at org.apache.hadoop.hive.ql.security.authorization.command.CommandAuthorizerV2.addHivePrivObject(CommandAuthorizerV2.java:210) [hive-exec-3.1.3000.7.1.7.0-551.jar:3.1.3000.7.1.7.0-551]

The create statement:

CREATE EXTERNAL TABLE default.test_delta
( id INT, activ INT)
STORED BY 'io.delta.hive.DeltaStorageHandler'
LOCATION '/dev/delta/tessttt';

It's anyone that know why this error occurs ?

Solution

  • You need to add in the properties of the table the following: TBLPROPERTIES('DO_NOT_UPDATE_STATS'='true');

    Example

    CREATE EXTERNAL TABLE default.test_delta
( id INT, activ INT)
STORED BY 'io.delta.hive.DeltaStorageHandler'
LOCATION '/dev/delta/tessttt'
TBLPROPERTIES('DO_NOT_UPDATE_STATS'='true';

    The solution also works for iceberg format. Inspired by: https://github.com/delta-io/connectors/issues/279