Search code examples
azure-synapsedbt

How to use CETAS (Synapse Serverless Pool) in dbt?

In Synapse Serverless Pool, I can use CETAS to create external table and export the results to the Azure Data Lake Storage.

CREATE EXTERNAL TABLE external_table
WITH (
    LOCATION = 'location/',
    DATA_SOURCE = staging_zone,  
    FILE_FORMAT = SynapseParquetFormat
)  
AS
SELECT * FROM table

It will create an external table name external_table in Synapse and write a parquet file to my staging zone in Azure Data Lake.

How can I do this in dbt?

Solution

  • I wrote a materialization for CETAS (Synapse Serverless Pool) here: https://github.com/intheroom/dbt-synapse-serverless

    It's a forked from dbt-synapse-serverless here: https://github.com/dbt-msft/dbt-synapse-serverless

    Also you can use hooks in dbt to use CETAS.