Search code examples
hadoophadoop2hortonworks-data-platformhortonworks-sandboxhortonworks-dataflow

How to copy a file from /user/maria_dev/tutorials/test.csv (HDP) to /sandbox/tutorial-files/640/nifi/input (HDF)?

Using Hortonworks HDP 3.0.1 and HDF Sandbox on Docker Desktop

I have logged into HDP, and tried to copy a file from /user/maria_dev/tutorials/test.csv point to HDF's /sandbox/tutorial-files/640/nifi/input BUT this does not work. How do I go copying a file from HDP to HDF this way with HDFS?

Solution

  • @jr134

    Using NiFi, you should configure connection to HDFS and get the files from HDP to HDF via nifi/hdfs processors. You will need to put the local HDP files into hdfs from local machine. Next you need to get copies of hdfs config files (core-site, hdfs-site, etc) and put them on the hdf cluster with permissions for nifi user to read/access. Once this is satisfied you need to make sure you have permissions from Nifi to HDFS by making sure nifi user has access to the files.