Search code examples
hivepartitionhive-metastorehive-partitionshiveddl

What are the allowed data types of partition column in hive?


I am pretty sure that complex types like STRUCT can not be the type of a partition column. But I am not sure if all the primitive types are valid or not. I have read a lot of documentation but didn't find anything.


Solution

  • Only primitive types are allowed. and should fit in the maximum file name limit (255- column_name_length-1 for equal sign, did not check exactly) in UNIX because partition is a folder.

    If you try to create table with complex type as a partition it fails:

    create table test_t(id int) partitioned by (somemap map<string, string>);
    

    FAILED: SemanticException [Error 10126]: Partition column must be of primitive type. Found somemap of type: map

    It allows to create table with primitive types in partition column, even with BINARY though not all characters are allowed in folder names and loading binary as is will cause failure.

    Everything can be serialized as string with allowed characters after all. And if you want, structs are possible to store as a strings and deserialized back though this is not practical because you need partition pruning to work and functions in the partition filter will disable this feature.