Search code examples
hadoopcygwinhbase

Trouble with cygwin and hadoop 2.6.1 - null\bin\winutils.exe

I want to run hadoop 2.6.1 on cygwin64. (Later I want to run hBsae,too) The problem is, that I start start-dfs.sh, I will get this exception:

 15/10/07 21:13:54 ERROR util.Shell: Failed to locate the winutils binary    in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
    at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
    at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:363)
    at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:438)
    at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:484)
    at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
    at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
    at org.apache.hadoop.hdfs.tools.GetConf.main(GetConf.java:331)

Okay, I know that this is a well known problem and some solutions in this forum say something about adding HADOOP_HOME to the envirements. But the problem is still available. this is a snippet of my bash_profile:

# cygwin path translation

export JAVA_HOME=$(cygpath -u $(cygpath -m -s "${JAVA_HOME}"))
export HADOOP_PREFIX=/home/user/hadoop-2.6.1

export PATH=$PATH:$(cygpath -u $(cygpath -m -s "${JAVA_HOME}\bin")):/home/user/hadoop-2.6.1/bin
export HADOOP_INSTALL=/home/user/hadoop-2.6.1

export HADOOP_HOME=/home/user/hadoop-2.6.1/
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=/home/user/hadoop-2.6.1/lib/native
export PATH=$PATH:$HADOOP_INSTALL/sbin
export PATH=$PATH:$HADOOP_INSTALL/bin

an echo $HADOOP_HOME also prints the path. But why is the path of winutils incomplete? Or is there a variable that I ferget?

greetings

Solution

  • You should set up winutils.exe in your PATH (environment variable).