Search code examples
pythonpandasdatabricksazure-databricksazure-data-lake

Databrics save pandas dataframe as CSV Azure Data Lake


I have pandas dataframe in the Azure Databricsk. I need to save it as ONE csv file on Azure Data Lake gen2.

I've tried with :

df.write.mode("overwrite").format("com.databricks.spark.csv").option("header","true").csv(dstPath)

and

df.write.format("csv").mode("overwrite").save(dstPath)

but now I have 10 csv files but I need one file and name it.

Thanks in advance.


Solution

  • I've found a solution :

    df.to_csv('/dbfs/mnt/....../df.csv', sep=',', header=True, index=False)