Search code examples
databricksazure-databricks

Download files (databricks/driver)

I tried to download an excel file that I generated via pandas but I can't find it ... I know it is in the file:/databricks/driver but I can download it ...

Is it possible to transfer it into storage or transfer it to my machine local?

I tried it but it didn't work.

dbutils.fs.cp('file:/databricks/driver/test.xlsx','dbfs:/mnt/datalake/test.xlsx')
Solution

  • Note: Using Databricks GUI, you can download full results (max 1 millions rows).

    enter image description here

    OR

    Using Databricks CLI:

    To download full results (more than 1 million), first save the file to dbfs and then copy the file to local machine using Databricks CLI as follows.

    dbfs cp "dbfs:/FileStore/tables/AA.csv" "A:\AzureAnalytics"

    Reference: Databricks file system

    The DBFS command-line interface (CLI) uses the DBFS API to expose an easy to use command-line interface to DBFS. Using this client, you can interact with DBFS using commands similar to those you use on a Unix command line. For example:

    # List files in DBFS
dbfs ls
# Put local file ./apple.txt to dbfs:/apple.txt
dbfs cp ./apple.txt dbfs:/apple.txt
# Get dbfs:/apple.txt and save to local file ./apple.txt
dbfs cp dbfs:/apple.txt ./apple.txt
# Recursively put local dir ./banana to dbfs:/banana
dbfs cp -r ./banana dbfs:/banana

    Reference: Installing and configuring Azure Databricks CLI