Search code examples
hadoopconfigurationhivetez

Hive - How to know which execution engine I am currently using

I want to automate my hive ETL workflow in such a way that I need to execute hive jobs on the basis of execution engine (Tez or MR) because of memory constraints.

Would you please help, as I wanted to cross-check in-between of my whole work-flow which execution engine currently I'm dealing with.

Thanks in advance.

Solution

  • The Hive execution engine is controlled by hive.execution.engine property. It can be either of the following:

    • mr (Map Reduce, default)
    • tez (Tez execution, for Hadoop 2 only)
    • spark (Spark execution, for Hive 1.1.0 onward).

    The property can be read & updated using hive/beeline cli

    • For reading - SET hive.execution.engine;
    • For updating - SET hive.execution.engine=tez;

    If you want to programmatically get this value out, you must go for HiveClient which supports multiple ways like JDBC, Java, Python, PHP, Ruby, C++, etc.

    References