Search code examples
javahivehive-metastore

How to check if a partition exists in Hive?

I have a Hive table, which is partitioned by column dt. I need to add a partition if it does not exists, for exmaple, dt='20181219'.

Now I'm using HiveMetaStoreClient#getPartition(dbName, tableName, 20181219). If the partition does not exists, then catch NoSuchObjectException and add it.

Is there any elegant way to achieve this in Java?

Solution

  • Use add_partition(Partition, ifNotExists, needResults) (javadoc) ... which (if the 2nd argument is true) will only create an partition if it doesn't already exist.

    Alternatively, just use add_partition(Partition) to add the partition without a test, and catch the AlreadyExistsException if it occurs.

    Any scheme that involves testing that an action is possible and then doing that action has a potential race condition. In between the "test" and the "do", some other agent (thread, client, whatever) could do an action (the same one or a different one) that will cause your attempt to fail.

    So your current approach is not only ugly. It also has a potential race condition if partitions could be created by multiple agents.