Terraform配置Azure数据收集规则KQL查询报错求助

问题场景 #

通过Terraform创建包含自定义表数据源的Azure数据收集规则（DCR），使用transformKql参数的KQL查询格式化数据。Azure Portal中可正常运行的KQL查询，转义后写入Terraform文件或使用Heredoc格式均报错，提示Undefined symbol: RawData。

Portal中可用的KQL查询 #

source | extend d=todynamic(RawData) | extend TimeCreated = extract(@"\d*[-]\d*[-]\d*\s\d*[:]\d*[:]\d*[.]\d*",0,RawData) | extend AppPriority = tostring(split(d,"-",3)) | extend ReqId = tostring(split(d,"-",4)) | extend Prog = tostring(split(d,"-",5)) | extend Msg = tostring(split(d,"-",6))

Terraform中转义后的查询 #

source | extend d=todynamic(RawData) | extend TimeCreated = extract(@\"\\d*[-]\\d*[-]\\d*\\s\\d*[:]\\d*[:]\\d*[.]\\d*\",0,RawData) | extend AppPriority = tostring(split(d,\"-\",3)) | extend ReqId = tostring(split(d,\"-\",4)) | extend Prog = tostring(split(d,\"-\",5)) | extend Msg = tostring(split(d,\"-\",6))

报错信息（已翻译） #

azurerm_monitor_data_collection_rule.dcr_name: Creating...
╷
│ Error: 创建数据收集规则(订阅: "11111111-2222-3333-4444-555555555555"
│ 资源组名称: "rg_name"
│ 数据收集规则名称: "dcr-name"): datacollectionrules.DataCollectionRulesClient#Create: 响应请求失败: StatusCode=400 -- 原始错误: autorest/azure: 服务返回错误. Status=400 Code="InvalidPayload" Message="数据收集规则无效" Details=[{"code":"InvalidTransformQuery","message":"编译查询时出错: SemanticError:0x00000006 at 1:30 : Undefined symbol: RawData","target":"properties.dataFlows[0]"}]
╵

解决方案 #

核心原因 #

在DCR的transformKql转换阶段，自定义日志数据源的原始内容字段是**Record**，而非RawData。RawData是日志数据写入Log Analytics表后才会生成的字段，转换阶段无法直接引用。

修正后的KQL查询 #

将所有RawData替换为Record：

source | extend d=todynamic(Record) | extend TimeCreated = extract(@"\d*[-]\d*[-]\d*\s\d*[:]\d*[:]\d*[.]\d*",0,Record) | extend AppPriority = tostring(split(d,"-",3)) | extend ReqId = tostring(split(d,"-",4)) | extend Prog = tostring(split(d,"-",5)) | extend Msg = tostring(split(d,"-",6))

Terraform中的推荐写法 #

使用Heredoc格式避免繁琐转义，同时保证代码可读性：

resource "azurerm_monitor_data_collection_rule" "dcr_name" {
  name                = "dcr-name"
  resource_group_name = "rg_name"
  location            = "eastus"

  data_flow {
    streams      = ["Custom-MyCustomTable_CL"]
    destinations = ["logAnalyticsDestination"]

    transform_kql = <<KQL
source | extend d=todynamic(Record) | extend TimeCreated = extract(@"\d*[-]\d*[-]\d*\s\d*[:]\d*[:]\d*[.]\d*",0,Record) | extend AppPriority = tostring(split(d,"-",3)) | extend ReqId = tostring(split(d,"-",4)) | extend Prog = tostring(split(d,"-",5)) | extend Msg = tostring(split(d,"-",6))
KQL
  }

  // 数据源、目标等其他配置省略...
}

验证要点 #

• 确认transform_kql中使用Record而非RawData
• Heredoc格式下无需额外转义双引号和反斜杠，Terraform会直接传递原始查询内容给Azure API
• 确保streams字段与自定义表的流名称匹配（格式通常为Custom-<TableName>_CL）

内容的提问来源于stack exchange，提问作者Sean Brinker