Search code examples
hbaseapache-zookeeperkerberos

HBase Zookeeper AUTH_FAILED - Failed to find any Kerberos tgt

Environment

  • HBase 1.5
  • Hadoop 2.9.2
  • Zookeeper 3.5.6

Error

Receiving the following error after configuring Zookeeper to use Kerberos and configuring a HBase jaas.conf login config

... in hbase-master.log

ERROR org.apache.zookeeper.ClientCnxn: SASL authentication with Zookeeper Quorum member failed:
 javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException:
 javax.security.sasl.SaslException: GSS initiate failed 
               [Caused by GSSException: No valid credentials provided 
                (Mechanism level: Failed to find any Kerberos tgt)]) 
               occurred when evaluating Zookeeper Quorum Member's  received SASL token. 
               Zookeeper Client will go to AUTH_FAILED state.

HBase jaas.conf

 Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    useTicketCache=true
    storeKey=true
    keyTab="/etc/security/keytabs/hbase.keytab"
    principal="hbase/@REALM.COM";
 };

hbase-env.sh

export HBASE_OPTS="-Djava.security.auth.login.config=/opt/hbase/conf/jaas.conf"
Solution

  • The problem is in the hbase-env.sh, hbase needs more than java.security.auth.login.config set in HBASE_OPTS.

    Correct way to configure Zookeeper jaas.conf:

    export HBASE_SERVER_JAAS_OPTS="-Djava.security.auth.login.config=/opt/hbase/conf/jaas.conf"
export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Djava.security.auth.login.config=/opt/hbase/conf/jaas.conf"

    If you have separate keytabs between your master and region you'll need two JAAS files and you must specify both

    • HBASE_SERVER_JAAS_OPTS
    • HBASE_MASTER_OPTS

    If you use only 1 kerberos principal for all of hbase you only need to set HBASE_SERVER_JAAS_OPTS