Search code examples
apache-sparkhadooppysparkhadoop-yarn

Hadoop getmerge fails when trying to merge files to local directory


I am trying to merge two files from my HDFS to a folder on my local machine's desktop. The command that I am using is:

hadoop fs -getmerge -nl /user/hadoop/folder_name/ /Desktop/test_files/finalfile.csv

But that returns the following error:

getmerge: Mkdirs failed to create file:/Desktop/test_files (exists=false, cwd=file:/home/hadoop)

Does anyone know why this might be? I couldn't find much of anything else in my search.


Solution

  • You need to create the local folder /Desktop/test_files/