Search code examples
snowflake-cloud-data-platformgreat-expectations

Case sensitive table/column names in Snowflake datasource for Great Expectations


I’ve noticed that my Snowflake expectations only work when the column parameter is written in lowercase (even if the original column is in uppercase):

"expectations": [
    {
        "expectation_type": "expect_column_values_to_be_unique",
        "kwargs": {
            "column": "email"
        }
    }
]

But when column is written as EMAIL I get the following error:

great_expectations.exceptions.exceptions.MetricResolutionError: Error: The column "EMAIL" in BatchData does not exist.

My Data Context config looks like this:

{
  "datasources": {
    "my_datasource": {
      "class_name": "Datasource",
      "module_name": "great_expectations.datasource",
      "execution_engine": {
        "class_name": "SqlAlchemyExecutionEngine",
        "connection_string": "snowflake://..."
      },
      "data_connectors": {
        "configured_asset_data_connector": {
          "class_name": "ConfiguredAssetSqlDataConnector",
          "assets": {
            "my_table_asset": {
              "table_name": "SOME_TABLE",
              "schema_name": "PUBLIC"
            }
          }
        }
      }
    }
  },
  "stores": {...}
}

According to this Pull Request, it’s possible to set the flag use_quoted_name to True in order to preserve case in table/column names but I couldn’t find any docs where to set this configuration. I'm using Great Expectations 0.15.41.

Does anybody know where to set this property?


Solution

  • This is now solved with PR #6951 without the need to set use_quoted_name