基于正则的Splunk条件搜索：筛选访问URL总数小于指定值的日志

日志格式示例 #

[2023-03-24 12:38:38,774] [INFO] - Tool - Finished scan cycle #20 for top URL = https://www.website.com/tool/ . Channel = mobile . Duration = 2h 52m 50s (seconds = 10370) . Total visited URLs = 3527. Max depth reached = 6.

[2023-03-24 09:45:38,482] [INFO] - Tool - Finished scan cycle #19 for top URL = https://www.website.com/tool/ . Channel = mobile . Duration = 2h 36m 10s (seconds = 9370) . Total visited URLs = 3326. Max depth reached = 6.

原搜索语句 #

index="xxxxx"  sourcetype="xxxxx"  host="xxxxx" AND "Finished scan cycle" AND "/tool" 
| timechart  min(seconds)

需求说明 #

需要更新搜索语句，仅保留Total visited URLs数值小于自定义值（例如3000）的日志，已写出正则表达式URLs\s=\s(?<value>\d+).，但不清楚如何在Splunk中应用实现数值筛选。

解决方案 #

通过rex提取字段、eval转换数值类型、where筛选三步即可实现需求，完整搜索语句如下：

index="xxxxx" sourcetype="xxxxx" host="xxxxx" "Finished scan cycle" "/tool"
| rex "Total visited URLs\s=\s(?<total_visited_urls>\d+)\."
| eval total_visited_urls = tonumber(total_visited_urls)
| where total_visited_urls < 3000
| timechart min(seconds)

步骤解释 #

• rex命令：基于正则提取Total visited URLs对应的数值，存储到total_visited_urls字段中（补充前缀Total visited 可避免匹配日志中其他含URLs的字段）
• eval tonumber：将提取出的字符串类型数值转换为数字类型，确保后续能进行大小比较
• where命令：筛选出total_visited_urls小于指定值（示例为3000）的事件
• 保留原有的timechart命令，继续执行统计逻辑

内容的提问来源于stack exchange，提问作者paulalexandru