I installed Apache kylin with following versions of following technologies: -
Apache Hadoop: - 2.7.3
Apache Hive: - 2.1.0
Apache Hbase:- 1.2.4
Apache Kylin: - 1.6
I am able to load data in Apache Kylin using sample.sh file given with the setup. But, when I try to build cubes for a time range, I get the following error during Step 3 of 'Build Cube' Process
File does not exist:
hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/hduser341814501/.staging/job_local341814501_0007/libjars/hive-exec-2.1.0.jar
Wherein hduser is the user who has started the kylin and is trying to build the cubes as well.
I checked the directory structure and see that no directory or file is present in the app/hadoop/tmp/mapred/staging/hduser341814501/.staging folder.
Does kylin try to create this directory structure and put hive-exec-2.1.0.jar in that directory structure and is unable to do so because of some permission issues?
Or is it coming because of some other error?
Though, the jar kylin is looking for is present in the hdfs://localhost:54310/usr/local//hivr/lib folder. Is it also possible that kylin is looking for the jar at wrong location?
Can anyone help me with what am I doing wrong?
Kylin will seek the hive jars from classpath, and then specify it with "tmpjars" property when submitting MR job. Please check your environment with:
1) run $KYLIN_HOME/bin/find-hive-dependency.sh, check the ouput;
2) search "Hive Dependencies After Filtered" in $KYLIN_HOME/logs/kylin.log, it should log the dependent jar that submitting to Hadoop job, like:
logs/kylin.log.2017-03-07:2017-03-07 11:58:18,772 INFO [pool-6-thread-16] common.AbstractHadoopJob:203 : Hive Dependencies After Filtered: /usr/hdp/2.5.4.0-121/hive/lib/hive-metastore-1.2.1000.2.5.4.0-121.jar,/usr/hdp/2.5.4.0-121/hive/lib/hive-exec-1.2.1000.2.5.4.0-121.jar,/usr/hdp/2.5.4.0-121/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.2.1000.2.5.4.0-121.jar
Also check whether the jars are existing on the machine that Kylin runs.