Search code examples
pythonhivethrifthue

Access Hive (Hiveserver2) Using Python

My whole goal is to manipulate Hive like everyone manipulate sqldb using MySQLdb in Python.

  • hue shell: I have no problem querying Hive table in hue shell, however, the hue shell is somehow a wrapper built on python with a lot of configurations added comparing with simple python. For example, tons of sys.path and os.environ has been added and even if I copied exactly the same path and environment variable when I open Python, it still will not work due to some error AttributeError: 'ConfigSection' object has no attribute 'HUE_PRINCIPAL'.

  • The method here works only for hiveserver1, I think. And the conversation from here keeps talking about get client generated by thrift. I have zero experience working with thrift and I am wondering am I on the wrong way, before diving into thrift.

Any help would be appreicated!

Solution

  • Finally, I use this package to do what I want.

    pyhs2