自动生成的dbt Python测试疑似硬编码导致恒失败问题咨询
DBT测试
expect_column_values_to_be_of_type失败问题 项目概况
我有一个Python DBT项目,通过YAML定义的数据模型如下:
version: 2 models: - name: company description: '' columns: - name: ID description: Unique identifier for the company entity. It is the company name tests: - unique - not_null - dbt_expectations.expect_column_values_to_be_of_type: column_type: VARCHAR - name: REMOTE_ID description: Company name tests: - dbt_expectations.expect_column_values_to_be_of_type: column_type: VARCHAR - name: CREATED_AT description: '' tests: - dbt_expectations.expect_column_values_to_be_of_type: column_type: TIMESTAMP_TZ - not_null - name: UPDATED_AT description: '' tests: - dbt_expectations.expect_column_values_to_be_of_type: column_type: TIMESTAMP_TZ
执行流程与测试失败信息
执行dbt compile、dbt run后运行dbt test,出现测试失败,控制台输出:
17:59:30 Failure in test dbt_expectations_expect_column_values_to_be_of_type_company_CREATED_AT__TIMESTAMP_TZ (models/fizzbuzz/domain/company.yml) 17:59:30 Got 1 result, configured to fail if != 0 17:59:30 17:59:30 compiled Code at target/compiled/myapp/models/fizzbuzz/domain/company.yml/dbt_expectations_expect_column_837854ce21e79ee1abe42f69891c68e9.sql
编译后的测试SQL内容
查看编译后的SQL文件,内容如下:
WITH relation_columns AS (SELECT Cast('ID' AS VARCHAR) AS relation_column, Cast('VARCHAR' AS VARCHAR) AS relation_column_type UNION ALL SELECT Cast('REMOTE_ID' AS VARCHAR) AS relation_column, Cast('VARCHAR' AS VARCHAR) AS relation_column_type UNION ALL SELECT Cast('CREATED_AT' AS VARCHAR) AS relation_column, Cast('TIMESTAMP_NTZ' AS VARCHAR) AS relation_column_type UNION ALL SELECT Cast('UPDATED_AT' AS VARCHAR) AS relation_column, Cast('TIMESTAMP_NTZ' AS VARCHAR) AS relation_column_type), test_data AS (SELECT * FROM relation_columns WHERE relation_column = 'CREATED_AT' AND relation_column_type NOT IN ( 'TIMESTAMP_TZ' )) SELECT * FROM test_data
疑问
我理解dbt compile会根据YAML定义生成上述测试SQL,dbt test执行该SQL判断测试是否通过。但该测试逻辑未关联实际的company数据库表,而是使用硬编码的CTE,其中CREATED_AT的类型被设为TIMESTAMP_NTZ,测试条件为类型不等于TIMESTAMP_TZ时返回结果,这会导致测试必然失败。请问这是DBT的bug,还是我对DBT测试机制存在误解?
内容的提问来源于stack exchange,提问作者hotmeatballsoup
相关产品推荐
相关产品推荐

