Search code examples
dagster

Environment variables in Dagster config YAML

I'm attempting to provide an environment variable in a config YAML file as such:

resources:
  be_warehouse:
    config:
      conn_str:
        env: DB_CONN_STR
  analytics_warehouse:
    config:
      conn_str:
        env: WH_DB_CONN_STR

but I am receiving the following error:

Invalid scalar at path root:resources:analytics_warehouse:config:conn_str. Value "{'env': 'WH_DB_CONN_STR'}" of type "<class 'dict'>" is not valid for expected type "String".

I have seen this syntax used in this official example. Am I missing something obvious?

Solution

  • The env: ENV_VAR support is available for config schema which is typed as StringSource. If these are @resources you are creating you just need to declare config_schema={'conn_str': StringSource} instead of just using str.

    https://docs.dagster.io/_apidocs/config#dagster.StringSource