Splunk转义字符串正则提取字段问题：amountTotal始终为空

问题场景 #

有一段带转义字符串的JSON格式日志，需要提取requestId、paymentBankType和amountTotal（对应currencyAmount的值）三个字段，当前编写的Splunk查询中，amountTotal始终返回空值，且需要处理数百万条结构可变的JSON数据。

原查询语句 #

index=test "mySearch" | eval _raw=replace(_raw,"\\\"","\"") 
| rex "requestId\"\:\"(?<requestID>([^"]+))"
| rex "paymentBankType\"\:\"(?<paymentBankType>([^"]+))"
| rex "amountTotal\"\:\{\"currencyAmount\"\:(?<amountTotal>([^"]+))"
| table requestID, paymentBankType, amountTotal

示例Payload #

{"eventId":"430215f5-d6e5-4c3d-b1e0-7f36dc22f76d","eventType":"TEST","eventTime":"2023-03-21T15:04:14.479+0000","eventPayload":{"requestId":"100000023679171","paymentMethod":[{"paymentMethodId":"e17579d2-51d6-4b03-a910-49886dcfa21a","paymentBankType":"AMEX","paymentType":"CREDITCARD","amountTotal":{"currencyAmount":312.11,"currencyUnit":"USD"}}]

问题原因 #

原查询中amountTotal的正则表达式错误：currencyAmount的值是无引号的数字（如312.11），但正则用[^"]+（匹配非引号字符）来捕获，自然无法匹配到数字内容。另外，用rex硬解析JSON在结构可变时容易失效，不如用Splunk原生的JSON解析工具可靠。

解决方案 #

方案1：用spath原生解析JSON（推荐，适配结构可变场景）

spath是Splunk专门用来解析XML/JSON的命令，能自动识别嵌套结构和数组，处理结构可变的JSON更稳定。修改后的查询：

index=test "mySearch" 
| eval _raw=replace(_raw,"\\\"","\"") 
| spath input=_raw 
| rename eventPayload.requestId as requestID, eventPayload.paymentMethod{}.paymentBankType as paymentBankType, eventPayload.paymentMethod{}.amountTotal.currencyAmount as amountTotal
| table requestID, paymentBankType, amountTotal

• spath input=_raw：解析整个_raw字段的JSON结构
• eventPayload.paymentMethod{}：{}用来处理数组类型的paymentMethod，自动展开数组中的每个元素对应的字段
• rename：把嵌套字段重命名为需要的字段名

方案2：修正rex正则表达式

如果坚持用正则，把amountTotal的正则改成匹配数字（含小数）的表达式：

index=test "mySearch" | eval _raw=replace(_raw,"\\\"","\"") 
| rex "requestId\"\:\"(?<requestID>[^\"]+)"
| rex "paymentBankType\"\:\"(?<paymentBankType>[^\"]+)"
| rex "amountTotal\"\:\{\"currencyAmount\"\:(?<amountTotal>\d+\.?\d*)"
| table requestID, paymentBankType, amountTotal

• \d+\.?\d*：匹配整数或小数，能覆盖如312.11、100、0.5这类数值格式

内容的提问来源于stack exchange，提问作者rickygrimes