Databricks error while trying to create delta table on ADLS Gen2

Question

I am working with Azure Databricks to create a delta table on Azure Data Lake Storage Gen2 and running into error.

Code:

dataframe.write.format("delta").mode("overwrite").option("path","abfss://<ContainerName>@<StorageAccount>.dfs.core.windows.net").saveAsTable("test_table")

Error:

IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: abfss://<ContainerName>@<StorageAccount>.dfs.core.windows.net_delta_log

Answer 1

You need to set path to the specific directory inside your Data Lake, not onto the top of the container. Something like:

ContainerName = "container"
StorageAccount = "account"
table_path = f"abfss://{ContainerName}@{StorageAccount}.dfs.core.windows.net/test-table"
dataframe.write.format("delta").mode("overwrite")\
  .option("path", table_path).saveAsTable("test_table")