Search code examples
google-bigqueryjinja2dbt

run dbt generic tests with tags attached to a specific model

I've been asked to attach generic DBT tests for BigQuery to models in the YAML definition. This is because the automation of the tests execution relies on tags for models and the need is to link a model name to a test failure.

These tests I wrote regard the entire model. They are not tied to a specific column.

I see slight variations of these test execution errors, it seems DBT cannot find the tests:

11:31:06  1 of 1 START test foo_thing_my_model_ ............... [RUN]
11:31:06  1 of 1 ERROR foo_thing_my_model_ .................... [ERROR in 0.02s]
11:31:06  
11:31:06  Finished running 1 test in 0 hours 0 minutes and 0.67 seconds (0.67s).
11:31:07  
11:31:07  Completed with 1 error and 0 warnings:
11:31:07  
11:31:07    Compilation Error in test foo_thing_my_model_ (models/<path-to>/my_model.yml)
  'test_foo_thing' is undefined. This can happen when calling a macro that does not exist. Check for typos and/or install package dependencies with "dbt deps".

I cannot see what is wrong with the setting below?

YAML config for the model:

version: 2
models:
  - name: my_model
    config:
      tags:
        - aaa
        - bbb
        - foo # this filter is expected to be picked up in the test execution
    tests:
      - foo_thing

The test is defined in the tests/ folder with filename foo_thing.sql and content (I let the model name to be injected as a parameter for the macro as described in https://docs.getdbt.com/best-practices/writing-custom-generic-tests and then used as {{ model }} although I would prefer if possible to reference the model with {{ ref("my_model") }}):

{% test foo_thing(model) %}
...
-- very complex SQL query that should return no rows to mark the test successful
...
{% endtest %}

I've also tried to place the test in ./tests/my_model/foo_thing.sql with no success.

I would like to invoke the test with this command: dbt test --select tag:foo

What's wrong with this setting? How can I fix it?

Solution

  • The test file needs to be nested in a directory called generic; so, in your case:

    ./tests/generic/test_foo_thing.sql

    The other valid place for tests (legacy, but not yet officially deprecated) is in the macros directory:

    ./macros/test_foo_thing.sql

    (It is possible that your dbt_project.yml config file defines different paths for tests and macros, but this is unlikely. If it does, change the paths above to match your config)