Search code examples
hadoophivekylin

Apache Kylin: Intermediate table not found

I’m a newbie for kylin. After installing, I run sample.sh, and then build the cube, but get the wrong message:

java.io.IOException: NoSuchObjectException(message:default.kylin_intermediate_kylin_sales_cube_desc_19700101000000_20160101000000_38b1539f_1f69_406d_89ed_96f3ca776841 table not found)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
     at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:101)
     at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:77)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
     at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
     at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
     at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:51)
     at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
     at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:130)
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
     at java.lang.Thread.run(Thread.java:745)
Caused by: NoSuchObjectException(message:default.kylin_intermediate_kylin_sales_cube_desc_19700101000000_20160101000000_38b1539f_1f69_406d_89ed_96f3ca776841 table not found)
     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1569)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:106)
     at com.sun.proxy.$Proxy49.get_table(Unknown Source)
     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1008)
     at org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:191)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105)
     at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
     at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
     ... 13 more

`

Issue 975 suggests kylin.job.hive.database.for.intermediatetable = default. After that, but I also get this error message.

When I run desc formatted kylin_intermediate_kylin_sales_cube_desc_1970... command in hive shell, I can get its formatted info. It demonstrates that this table exists in Hive. Why could not Kylin load this table from Hive?

Kylin version = 1.2
Hive version = 0.13.1-cdh5.3.2
Hbase version = 0.98.6+cdh5.3.2
Hadoop version = 2.5.0-cdh5.3.2
Solution

  • I have solved this problem. The reason is that hive.metastore.uris property is not set in hive-site.xml. Kylin uses HCatalog to read Hive table. HCatalog will use hive.metastore.uris property to create HiveMetaStoreClient and get table meta. This page has detailed explanation.

    <property>
  <name>hive.metastore.uris</name>
  <value>thrift://localhost:9083</value>
  <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>
</property>

    To start metastore using command:

    nohup hive --service metastore -p 9083 &