Search code examples
hadoophdfsflume

Move zip files from one server to hdfs?

What is the best approach to move files from one Linux box to HDFS should I use flume or ssh ?

SSH Command: 

cat kali.txt | ssh user@hadoopdatanode.com "hdfs dfs -put - /data/kali.txt"

Only problem with SSH is I need to mention password every time need to check how to pass password without authentication.

Can flume move files straight to HDFS from one server?

Solution

  • Maybe you can make passwordless-ssh, then transfer files without entering password