Search code examples
hbaseclouderacloudera-cdhhfile

Cloudera CDH 5.7.2 / HBase: How to Set hfile.format.version?


With CDH 5.7.2-1.cdh5.7.2.po.18, I am trying to use Cloudera Manager to configure HBase to use visibility labels and authorizations, as described in the Cloudera Community post below:

Cloudera Manager Hbase Visibility Labels

Using Cloudera Manager, I have successfully updated the values of the following properties:

  • hbase.coprocessor.region.classes: Set to org.apache.hadoop.hbase.security.visibility.VisibilityController
  • hbase.coprocessor.master.classes: Set to org.apache.hadoop.hbase.security.visibility.VisibilityController
  • hbase.security.authorization: Set to true

Unfortunately, for whatever reason, Cloudera Manager doesn't expose the hfile.format.version variable, so I am unable to set it to 3.

When I restart HBase after updating only the three aforementioned variables, errors such as the following appear in the role log:

ERROR org.apache.hadoop.hbase.coprocessor.CoprocessorHost: The coprocessor org.apache.hadoop.hbase.security.visibility.VisibilityController threw java.lang.RuntimeException: A minimum HFile version of 3 is required to persist security labels. Consider setting hfile.format.version accordingly.

I'd love to set hfile.format.version accordingly, but... how would I do that, when it's not exposed in Cloudera Manager? For example, is there some config file on the file system that I can modify? Cloudera Manager has to be getting the current settings from somewhere.

Thanks!


Solution

  • Cloudera Manager has a configuration section it calls the Safety Valve for exactly this purpose. It will let you set arbitrary properties to whatever value you need. See this doc for the following example:

    1. Go to the HBase service.
    2. Click the Configuration tab.
    3. Select Scope > RegionServer.
    4. Select Category > Advanced.
    5. Locate the HBase Service Advanced Configuration Snippet (Safety Valve) for hbase-site.xml property, or search for it by typing its name in the Search box.

    6. Add your custom XML (e.g.):

      <property> <name>property-name</name> <value>property-value</value> </property>