Search code examples
azure-data-lakedbeaverazure-data-lake-gen2duckdb

Is it possible to use DBeaver and the DuckDB connector to read parquet files directly from Azure Datalake Storage?


I've used DBeaver and the DuckDB database connection (Path = :memeory:) to read local .parquet files.

SELECT * FROM "C:\folder\myfile.parquet"

Is it possible to read .parquet files direct from ADLS?

I've tried the following: SELECT * FROM "abfs://[email protected]/folder/my.parquet" but that doesn't work

If possible, could someone please provide an example connection/path string?

Update It is possible, please follow the below example:

INSTALL azure;
LOAD azure;

SET azure_storage_connection_string = '<connection string from storage account/access keys/connection string>';

SELECT * FROM 'azure://<continer>/<folder>/<filename>.parquet' limit 10;

Solution

  • It is possible, please follow the below example:

    INSTALL azure;
    LOAD azure;
    
    SET azure_storage_connection_string = '<connection string from storage account/access keys/connection string>';
    
    SELECT * FROM 'azure://<continer>/<folder>/<filename>.parquet' limit 10;