I have pandas dataframe in the Azure Databricsk. I need to save it as ONE csv file on Azure Data Lake gen2.
I've tried with :
df.write.mode("overwrite").format("com.databricks.spark.csv").option("header","true").csv(dstPath)
and
df.write.format("csv").mode("overwrite").save(dstPath)
but now I have 10 csv files but I need one file and name it.
Thanks in advance.
I've found a solution :
df.to_csv('/dbfs/mnt/....../df.csv', sep=',', header=True, index=False)