ADF查找活动调用CDC函数返回NULL行,但SSMS执行正常的问题求助
ADF查找活动调用CDC函数返回NULL行,但SSMS执行正常的问题求助
大家好,我现在在基于Change Data Capture(CDC)搭建ADF增量更新管道,遇到了一个很奇怪的问题,想请各位帮忙分析下:
核心问题
我在ADF的Lookup活动里执行一段CDC相关的SQL脚本,返回的结果是一行所有字段全为NULL,但在SSMS里执行完全相同的脚本,能拿到3行我需要的正常数据。
ADF Lookup执行结果(JSON片段)
{ "count":1, "value":[ { "__$start_lsn": null, "__$operation": null, "__$update_mask": null, "Company_Code": null, "Equipment_Code": null, "id": null } ] }
SSMS执行结果
__$start_lsn __$operation __$update_mask Company_Code Equipment_Code id 0x000CBDE900022E98002C 4 NULL 011 50 15 0x000CBDE900054118001C 4 NULL 011 51 16 0x000CBDE9000541A0000C 4 NULL 011 52 17
已尝试的调试操作
为了定位问题,我做了以下尝试,但还是没找到根因:
- 把脚本改成统计行数:
SELECT COUNT(*) FROM cdc.fn_cdc_get_net_changes_my_custom_table(...),ADF能正确返回3 - 只查询单个int类型字段
id,ADF仍然返回一行id: null - 将脚本封装成存储过程调用,ADF执行后还是得到全NULL的结果
- 直接查询CDC的变更表
cdc.my_custom_table_CT,ADF能正常拿到所有变更数据 - 单独调用
sys.fn_cdc_get_min_lsn('my_custom_table')和sys.fn_cdc_max_lsn(),ADF返回的LSN值和SSMS里的一致,是正确的
我的ADF配置
测试管道的JSON配置
{ "name": "Test", "properties": { "activities": [ { "name": "Test", "type": "Lookup", "dependsOn": [], "policy": { "timeout": "0.12:00:00", "retry": 0, "retryIntervalInSeconds": 30, "secureOutput": false, "secureInput": false }, "userProperties": [], "typeProperties": { "source": { "type": "SqlServerSource", "sqlReaderQuery": "DECLARE @from_lsn binary(10), @to_lsn binary(10); \nSET @from_lsn =sys.fn_cdc_get_min_lsn('my_custom_table'); \nSET @to_lsn = sys.fn_cdc_get_max_lsn();\nSELECT * FROM cdc.fn_cdc_get_net_changes_my_custom_table(@from_lsn, @to_lsn, 'all')", "queryTimeout": "02:00:00", "partitionOption": "None" }, "dataset": { "referenceName": "TestDataset", "type": "DatasetReference" }, "firstRowOnly": false } } ], "annotations": [] } }
关联的数据集JSON配置
{ "name": "TestDataset", "properties": { "linkedServiceName": { "referenceName": "TestLinkedService", "type": "LinkedServiceReference" }, "annotations": [], "type": "SqlServerTable", "schema": [ { "name": "__$start_lsn", "type": "binary" }, { "name": "__$operation", "type": "int", "precision": 10 }, { "name": "__$update_mask", "type": "varbinary" }, { "name": "Company_Code", "type": "varchar" }, { "name": "Equipment_Code", "type": "varchar" }, { "name": "id", "type": "int", "precision": 10 } ], "typeProperties": { "schema": "cdc", "table": "my_custom_table_CT" } }, "type": "Microsoft.DataFactory/factories/datasets" }
现在我很困惑:既然直接查CT表没问题,单独调用LSN函数也没问题,为啥一用cdc.fn_cdc_get_net_changes_*这个函数,ADF就返回全NULL的行呢?有没有大佬遇到过类似的问题,或者能给我一些新的调试方向?
内容来源于stack exchange
相关产品推荐
相关产品推荐

