Search code examples
apache-sparkpysparkazure-synapsemicrosoft-fabric

Can't read files from OneLake shortcut to ADLS in Microsoft Fabric


I'm doing a first pass with Microsoft Fabric and can't read files with a notebook when querying files in a OneLake shortcut to an Azure Storage account.

I have configured the shortcut using Account Key for authentication.

When attempting to read a file, I receive the following error:

Py4JJavaError: An error occurred while calling xxx.parquet. : java.nio.file.AccessDeniedException: Operation failed: "Forbidden", 403

What am I doing wrong?


Solution

  • The issue was my storage account was not Azure Data Lake Gen2 enabled - it was a basic Azure Storage account. Slightly confused as the "Endpoints" section on the account provided me with a dfs option - guess this is included regardless of the underlying configuration of the storage account.

    I resolved this issue by upgrading my storage account to enable the hierarchical namespace.