Search code examples
ibm-cloudbiginsightsbiginsight-examples

How can I download a file using the WebHDFS knox groovy library?

The WebHDFS examples shows how to list files and folders in hdfs, make a directory and upload a file using BigInsights WebHDFS?

How can I adapt the examples to download a file for BigInsights WebHDFS?

Solution

  • The knox api documentation provides many more examples, e.g.

    import groovy.json.JsonSlurper
import org.apache.hadoop.gateway.shell.Hadoop
import org.apache.hadoop.gateway.shell.hdfs.Hdfs

gateway = "https://localhost:8443/gateway/sample"
username = "bob"
password = "bob-password"
dataFile = "README"

session = Hadoop.login( gateway, username, password )
text = Hdfs.get( session ).from( "/tmp/example/README" ).now().string
file = new File('README') 
file << text
session.shutdown()