Search code examples
hadoophadoop2

How to get a list of Java classes which are called at the time of running hadoop hdfs command?

Is there any command by which we can find what are all the java classes that would be called if we submit a command like:

hdfs dfs –copyFromLocal data.txt /tmp

Solution

  • As such there is no command to get Class diagram of internally called classes. But below is one sequence of classes invoked when any of the hadoop shell command is run on terminal.

    First class is org.apache.hadoop.fs.FsShell.java

    If you type hadoop fs or hdfs dfs without any option, printUsage method of FsShell.java is called to display help.

    On terminal you will get output of printUsage method like this. hadoop fs without arg output

    Second class is org.apache.hadoop.fs.shell.FsCommand, is called from within FsShell class.

    all the shell commands like (-ls)Ls.class, (-mkdir)Mkdir.class are registered with the help of CommandFactory.java in FsShell class.

    Eventually you have a list of hadoop commands classes like below.

    Few of them have some static inner classes in it for similar kind of functionality.

    (-ls,-lsr)org.apache.hadoop.fs.shell.Ls.java
(-mkdir)org.apache.hadoop.fs.shell.Mkdir.java
(-mv) org.apache.hadoop.fs.shell.MoveCommands.java
(-rm,rmdir,-rmr,-expunge)org.apache.hadoop.fs.shell.Delete.java

    And the celebrities of town, because everybody knows them i.e.copy commands.

    (-getmerge,-cp,-copyFromLocal,copyToLocal,-get,-put)org.apache.hadoop.fs.shell.CopyCommands.java