Search code examples
hadoophivehdfseventhandlerdata-lake

How to write Hive hook to access meta data level event changes by extending MetaStoreEventListener

I am interested in fetching hive meta data level events which are generally stored in hive server 2 - like event based triggers for partition addition, deletion, table addition and deletion, etc. There is already an abstract class MetaStoreEventListener but I am unable to find a solid implementation of the same to access all Meta data related informations. Can you guys give me some insight in the right directions ?

Solution

  • The best wake is to create your own handler utility or framework to wrap around the traditional NotificationListener class (which extends the MetaStoreEventListener class) in the hcatalog package.

    If you decide to solely use the NotificationListener class, then the following inbuilt listeners are readily available for usage:

    1. Partitions (Add-onAddPartition, Alter-onAlterPartition, Drop-onDropPartition)
    2. Databases (Create-onCreateDatabase, Drop-onDropDatabase)
    3. Tables (Create-onCreateTable, Alter-onAlterTable, Drop-onDropTable)

    Additionally, you can independently explore the HDFS package handlers. Or, on a more advanced side, write your own independent framework encapsulating the default event handlers provided by HCatalog and HDFS.