How to copy files from local to hdfs using oozie

Question

I am trying to copy files from my edge node to HDFS using oozie. Many suggested to setup password less ssh to get this done.

Iam unable to login to oozie user as it is a service user.

Is there any other way other than password less ssh.

Thanks in advance.

Answer 1

Other than password less ssh there are two more options : 1. My preferred option : Use JSch java library and create a java application which will accept a shell script to be executed as argument. Using the JSch , it will perform ssh on the configured edge node and execute the shell script on the edge node. In the jsch, you can configure, the edgenode username and password. Use 'JCEKS' file to store the password. Then add a Java Action in Oozie to run the java application created using JSch. 2. Use "/usr/bin/expect" library to create a shell script, which will perform ssh on edgenode and then run the configured shell script. More details are here Use expect in bash script to provide password to SSH command