Search code examples
pythondbt

How can I run a python script at the end of a specific DBT model, using the post_hook param within config()?


I would like to be able to run a python script only at the end of a specific DBT model. My idea is to use post_hook parameter from config() function of that specific model.

Is there a way to do this?


Solution

  • You cannot do this today. dbt does not provide a Python runtime.

    Depending on how you deploy dbt, you could use fal for this (either open source or cloud): https://fal.ai/, or another (heavier) orchestrator, like Airflow, Dagster, or Prefect.

    You should also know that there is an active Discussion about External Nodes and/or executable exposures that would solve for this use case: https://github.com/dbt-labs/dbt-core/discussions/5073

    dbt is also planning to release Python-language models in the near future, but that is unlikely to solve this use case; that Python will be executed in your Warehouse environment, and may or may not be able to make arbitrary web requests (e.g., Snowpark is really just dataframe-python that gets transpiled to SQL)