Search code examples
javabashhadoopapache-zookeeperaccumulo

How to start Hadoop, Accumulo, and ZooKeeper from a java program?


I am attempting to convert a bash script to a java program. Within this script I run the start scripts for Hadoop, Zookeeper, and Accumulo:

/hadoop/bin/start_all.sh  
/zookeeper/bin/zkServer.sh start  
/accumulo/bin/start_all.sh  

This is simple to do in a script. And if the programs are already running I can call these startup scripts again no issue and the programs will simply output that they are already running and their pids.

I'm trying to figure out if there is a way to do this within a java program. Is there some hidden command in the Hadoop/ZooKeeper/Accumulo API where I can run Class.run(configs) and it'll start or try to start Hadoop/ZooKeeper/Accumulo?

My next step is that I can probably use jsch to run ssh commands, but that seems like I'm not really leaving the bash script behind.

Edit: executing hadoop example jar files from java In this question the asker is executing the start commands using Runtime. Is this an appropriate way to start Hadoop? I'd rather use the native Hadoop API if there are commands to use there.


Solution

  • There isn't any specific API to start the Hadoop services or Zookeeper services in my view. Take for an instance the class org.apache.hadoop.hdfs.server.namenode.NameNode

    Though you can use the main() in the above class to start the service.

    From the script, Hadoop uses something like the following to start the services calling the main() function of the different classes with arguments.

    nohup $_JAVA_EXEC -Dproc_$COMMAND $JAVA_HEAP_MAX $HADOOP_OPTS -classpath "$CLASSPATH" $CLASS "$@" > "$_HADOOP_DAEMON_OUT" 2>&1 < /dev/null
    

    Where, CLASS = org.apache.hadoop.hdfs.server.namenode.NameNode Others are self explanatory.